📖Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench

Paper: Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench

Authors: Zheyuan Liu, Guangyao Dou, Mengzhao Jia, Zhaoxuan Tan, Qingkai Zeng, Yongle Yuan, Meng Jiang

Venue: NAACL 2025 (Long Papers)

URL: https://arxiv.org/abs/2410.22108

---

MLLMU-Bench

Introduction

Large Language Models and Multimodal Large Language Models acquire diverse knowledge as they are trained on large-scale data, but they may also memorize personal information or copyrighted content contained in that training data. Excluding the problematic data and retraining the model from scratch is the most direct approach, but the cost is prohibitive for large models, so machine unlearning, which removes only the influence of specific data, is being studied as an alternative.

In the LLM field, unlearning benchmarks such as TOFU and WMDP have been proposed. In MLLMs, however, there is a difference: the same knowledge can be learned together with images, not only with text. For instance, even if the textual information about a particular person is removed, it is hard to call the unlearning complete if the model can still answer related questions when shown that person’s face.

For MLLMs, therefore, it is necessary to evaluate not just one modality but whether visual-textual knowledge has actually been removed across multiple modalities. MLLMU-Bench (Multimodal Large Language Model Unlearning Benchmark) was proposed for this purpose.

Main Content

Dataset Construction

MLLMU-Bench comprises 653 profiles, 1,153 images, and 20,754 questions in total. 500 are fictitious profiles generated with GPT-4o, and the remaining 153 are real celebrity profiles. Synthetic images from ThisPersonDoesNotExist are used for the faces of the fictitious people, so that constructing the benchmark itself does not create privacy problems for real individuals.

Each profile includes information such as name, birthplace, occupation, and residence, and 14 question-answer pairs are constructed per profile. Of these, 7 are multimodal questions using image and text together, and the remaining 7 are text-only questions.

Because the same profile knowledge is queried through different modalities, it is also possible to check whether particular information was removed in only one modality.

Evaluation Datasets

MLLMU-Bench evaluates unlearning using four datasets: the Forget Set, Test Set, Retain Set, and Real Celebrity Set.

The Forget Set is the actual target of unlearning. 5%, 10%, and 15% of the 500 fictitious profiles are designated as the Forget Set, and the lower the performance on those profiles, the better the target information is considered to have been removed.

However, lower performance on the original questions and images alone does not establish that the knowledge itself has been removed. To check this, the Test Set paraphrases the questions for the same individuals and alters the pose and angle of the images with Arc2Face. In other words, the Forget Set evaluates unlearning efficacy and the Test Set evaluates unlearning generalizability.

Conversely, the Retain Set checks whether knowledge of fictitious profiles that are not unlearning targets is preserved, and the Real Celebrity Set evaluates the effect on neighboring knowledge.

Evaluation Tasks

Each dataset is evaluated with three tasks: Classification, Generation, and Cloze.

Classification asks about profile attributes such as birthplace or occupation in multiple-choice form and measures classification accuracy.

Generation produces free-form text, using ROUGE-L to measure textual overlap with the ground truth and a Factuality Score in which GPT-4o rates the factuality of the generated answer from 1 to 10.

The Cloze task provides only the name and partial context and asks the model to fill in specific information. Whether the generated answer exactly matches the ground truth is measured as cloze accuracy. Because profile information must be recovered from limited context, this can be seen as an evaluation of the memorized knowledge remaining inside the model.

On the Forget Set and Test Set, lower values of each metric indicate stronger forgetting; on the Retain Set and Real Celebrity Set, higher values indicate that model utility is better preserved.

Unlearning Baselines

The authors first fine-tune LLaVA-1.5-7B and Idefics2-8B on the 500 fictitious profiles, then apply existing LLM unlearning methods under the 5%, 10%, and 15% forget settings.

The simplest, Gradient Ascent (GA), optimizes the Forget Set loss in reverse.

That is, where ordinary training raised the likelihood of the correct answer, unlearning updates the parameters so that this likelihood decreases.

Gradient Difference (GD) considers the Retain Set loss as well as the Forget Set loss.

The aim is to remove the target knowledge on the Forget Set while maintaining performance on the Retain Set.

KL Minimization uses KL divergence so that the current model maintains an output distribution on the Retain Set similar to the pre-unlearning model. NPO places the Forget Set as dispreferred data and lowers the preference for the target answer in a preference-optimization form.

Experiments & Results

Baseline Comparison

Classification accuracy in the 5% forget setting on LLaVA-1.5-7B is as follows.

GA and Gradient Difference substantially lowered Forget Set performance, showing relatively strong forgetting. But performance on the Retain Set and Real Celebrity Set fell as well. Conversely, NPO and Prompting preserved existing knowledge relatively well but forgot comparatively weakly.

It is therefore hard to judge that simply lowering Forget Set accuracy further makes for better unlearning.

Multimodal vs. Unimodal Unlearning

MLLMU-Bench compares the same unlearning algorithm in a multimodal setting using image+text and a unimodal setting using text only.

No single modality was consistently superior across all tasks. In classification, multimodal unlearning showed stronger forgetting under image+text evaluation, but unimodal unlearning was stronger under pure-text evaluation. In generation and cloze as well, the unimodal approach showed generally stronger forgetting.

That said, the strong forgetting of unimodal unlearning sometimes came together with degraded performance on the Retain Set and Real Celebrity Set. Ultimately, input modality is not merely a change of data format; it also bears on which knowledge is removed and which tasks are affected.

Forgetting vs. Model Utility

The result confirmed repeatedly throughout the paper is the trade-off between forgetting and model utility.

GA is among the methods showing the strongest forgetting, but retain accuracy also drops considerably. Conversely, NPO and Prompting preserve utility relatively well but remove the target knowledge only weakly.

The authors examine utility not only on the Retain Set but also on the neighboring knowledge of the Real Celebrity Set, reasoning ability on MMMU, and helpfulness on LLaVA-Bench. Current baselines have not reached the ideal regime of removing only the target knowledge while leaving the remaining abilities intact.

Reproduction

To verify the paper’s method directly, some experiments were reproduced starting from the fine-tuned LLaVA_Vanilla checkpoint in the official MLLMU-Bench repository. Given the computational cost, the check focused on GA and Gradient Difference in the 5% forget setting.

In the official code as well, GA applies a negative sign to the answer-token NLL of the Forget Set to perform gradient ascent, corresponding directly to the paper’s objective.

By contrast, Gradient Difference showed a discrepancy in optimization between the paper objective and the official implementation.

The paper presents the following combined objective.

In the official code, however, the entire Forget Set is first processed with gradient ascent, and then the entire Retain Set is processed with gradient descent in a separate loop.

That is, the official code does not compute the Forget Set and Retain Set in a single loss but performs the two phases sequentially.

# Forget phase
loss_forget = -outputs.loss
accelerator.backward(loss_forget)
optimizer.step()

# Retain phase
loss_retain = outputs.loss
accelerator.backward(loss_retain)
optimizer.step()

In short, the paper objective expresses the forget loss and retain loss as a single optimization objective, whereas the official implementation updates on the entire Forget Set first and then trains again on the entire Retain Set.

Paper
Forget loss ↑ + Retain loss ↓
          ↓
   Combined objective

Official code
Forget Set → Gradient Ascent
               ↓
Retain Set → Gradient Descent

In the reproduction, the 5% setting had 96 iterations for the forget phase and 1,831 for the retain phase. The run itself completed, but the result is hard to regard as a reproduction under optimization conditions identical to the paper objective.

Conclusion

MLLMU-Bench extends text-based unlearning evaluation to the multimodal setting, aiming to assess whether private knowledge spanning images and text has actually been removed. In particular, it uses not only the Forget Set but also a Test Set of paraphrased questions and altered images, together with the Retain Set and Real Celebrity Set, to measure efficacy, generalizability, and utility.

Beyond that, the experiments established that results differ by task depending on the unlearning modality. Multimodal unlearning did not always remove multimodal knowledge better, and the strong forgetting of the unimodal approach could likewise come with utility degradation. Moreover, the method that lowered Forget Set performance the most was not always the best unlearning method, since the goal is to remove only the target knowledge while preserving unrelated knowledge and general capability.