📖Visual Adversarial Examples Jailbreak Aligned Large Language Models

Paper: Visual Adversarial Examples Jailbreak Aligned Large Language Models

Authors: Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal

Venue: AAAI 2024

URL: Visual Adversarial Examples Jailbreak Aligned Large Language Models

Introduction

A Visual Language Model (VLM) is a multimodal model that attaches a vision encoder to a Large Language Model (LLM) so that images, not just text, can be processed as input. Typically the vision encoder extracts visual features from the input image, and a connection module such as a projection layer converts them into an embedding form the LLM can understand. The LLM then generates a response using both the embedding obtained from the image and the text input.

Prior work on visual adversarial attacks mainly dealt with adding perturbation to an image so that it is misclassified.

Prior work on LLM jailbreaks, by contrast, focused on manipulating text input, through role-playing, prompt injection, or adversarial suffixes, to bypass the model’s safety alignment.

This paper connects the two lines of research and addresses the following research question.

If the image input of a VLM is adversarially perturbed, can the safety alignment of the attached LLM be bypassed?

The important difference here is the input space of images versus text. Text tokens are discrete, so a computed gradient cannot be applied directly to update the tokens themselves. Gradients with respect to embeddings can be computed, but changing the actual input requires a gradient-guided token search method such as AutoPrompt.

Image pixels, on the other hand, can be treated as continuous real values during optimization. The gradient of the loss can therefore be backpropagated all the way to the image pixels, optimizing the input directly.

In other words, adding image input extends the model’s capability while simultaneously adding a continuous, high-dimensional attack surface for an adversary to exploit. The paper’s main contributions can be summarized as follows.

1. It extends visual adversarial examples into an input for bypassing the safety alignment of a VLM.
2. It shows that a single adversarial image can generalize to a variety of harmful instructions not used during optimization.

Attack Method

Threat Model

The paper assumes a one-turn setting consisting of a single user-model interaction.

The attacker generates an adversarial image $x_{adv}$ and, at inference time, feeds this image to the model together with a new harmful text prompt $x_{harm}$. This can be expressed as follows. \(p_\theta(y|[x_{adv},x_{harm}])\) The goal of the attack is not an input that works on one specific harmful instruction, but a single universal visual adversarial example that can be reused with a variety of harmful instructions not seen during optimization.

The paper’s main attack assumes a white-box model in which the attacker can access the model architecture and parameters and compute gradients.

Image Tuning

The attack principle in this paper is similar to prompt tuning.

Prompt tuning learns a continuous soft prompt embedding with the model parameters frozen, guiding the model to perform a specific task better. The key point is that instead of fine-tuning the whole model, the prompt passed as input is optimized.

Similarly, this paper freezes the model parameters and optimizes only the image pixels to change the image embedding produced by the visual encoder. The optimized image embedding acts like a soft embedded prompt prepended to the LLM’s input, steering the VLM toward generating a harmful response.

Few-shot Harmful Corpus

To carry out the attack, a small harmful target corpus is constructed as follows. \(Y = \{y_i\}_{i=1}^{m}\) Here $y_i$ is a harmful target output sentence that the attacker wants the model to generate.

The paper uses 66 harmful sentences in total. They were produced by asking an early LLaMA-1 model, which had no safety alignment, for offensive sentences about particular groups or humanity, after which humans manually removed duplicates and non-harmful results.

The sentences break down as follows:

1. 21 sentences about a particular gender group
2. 28 sentences about particular racial groups
3. 17 sentences about humanity as a whole

This corpus is used only during attack optimization; the subsequent evaluation uses harmful instructions not directly contained in the corpus, in order to measure the attack’s generalization.

Optimization Objective

The attacker maximizes the probability that the 66 harmful target outputs are generated when the adversarial image is fed in. \(\mathbf{x}_{\mathrm{adv}}^{*} = \arg\min_{\mathbf{x}_{\mathrm{adv}}\in\mathcal{B}} -\sum_{i=1}^{m} \log p_{\theta} \left( \mathbf{y}_{i}\mid\mathbf{x}_{\mathrm{adv}} \right)\) Here

• $x_{adv}$ is the adversarial image being optimized.
• $y_i$ is a fixed harmful target output.
• $p_\theta(y \mid x)$ is the probability that a VLM with parameters $\theta$ generates output $y$ conditioned on input $x$.
• $B$ is the constraint set the attack image must satisfy. In the constrained attack, the $\ell_\infty$ distance from the original image and the valid pixel range are limited. In the unconstrained attack, no distance constraint from the original image is applied, but pixel values must still lie within the valid range $[0,1]$.

The objective is to minimize the negative log-likelihood of the harmful target outputs (gradient descent).

What the attack directly optimizes here is the token-level generation loss of the harmful target sentences.

Optimization Method

The paper uses Projected Gradient Descent (PGD) to implement the image attack. \(\mathbf{x}_{\mathrm{adv}}^{(t+1)} = \Pi_{\mathcal{B}} \left( \mathbf{x}_{\mathrm{adv}}^{(t)} - \alpha\, \operatorname{sign} \left( \nabla_{\mathbf{x}_{\mathrm{adv}}^{(t)}}\mathcal{L} \right) \right)\)

• $t$: optimization iteration
• $\alpha$: the step size by which pixels are changed in one update
• $\nabla_{x_{adv}}\mathcal{L}$: the loss gradient with respect to the image pixels

At each iteration, 8 of the 66 target sentences are sampled and their negative log-likelihood is computed. The gradient of the loss is then backpropagated through every module of the VLM down to the image pixels, and the pixels are updated in the direction that decreases the loss.

Evaluating the Attack

The paper evaluates the attack against the following three VLMs.

• MiniGPT-4 13B
• InstructBLIP 13B
• LLaVA

Evaluation datasets and metrics by task

For human evaluation, 40 human-written harmful text instructions are used, spanning the four categories of identity attack, disinformation, violence/crime, and X-risk.

For each instruction, 10 responses are generated with nucleus sampling ($p=0.9$, temperature $=1$). Whether each response produced harmful content matching that instruction is then assessed manually, and the average attack success rate per category is measured.

• RealToxicityPrompts (challenging subset: 1,225 prompts)
  • Evaluation method: Perspective API and the Detoxify classifier

Model settings / hardware

• Step size: $\alpha=1/255$
• Optimization iterations: $5000$
• GPU: NVIDIA A100 80GB

The image pixel perturbation budget was set to $\epsilon=16/255, 32/255, 64/255$. Separately, an unconstrained attack applying no distance constraint from the original image was also evaluated.

Performance under Human Evaluation

The human evaluation results for MiniGPT-4 are as follows.

The values are the average attack success rate (%) per category. Success is judged by whether harmful content matching the harmful instruction appeared in each generated response.

The visual attack substantially amplified the probability of carrying out the harmful instruction across all four categories relative to the baseline image.

One interesting point is that it does not always increase linearly with $\epsilon$; in many cases $\epsilon=32/255$ gives the highest performance (though this varies by benchmark). This shows that a larger perturbation budget does not always lead to better jailbreak performance.

Performance on RealToxicityPrompts

The RealToxicityPrompts results are as follows.

Looking at the Perspective API metric, the $\epsilon=64/255$ adversarial attack increased the rate from 34.8% to 67.2% relative to an unperturbed image.

Although the corpus used for attack optimization is limited in scope, the evaluation showed increased output rates across several toxicity attributes. Note, however, that an increase in the toxicity detection rate and the success rate of carrying out harmful instructions are different metrics and must be interpreted separately.

Measuring Transferability across Models

The paper optimized image pixels against one VLM and then fed the resulting adversarial image to other VLMs to evaluate black-box transferability.

Performance was measured with RealToxicityPrompts + Perspective API, giving the following results.

Diagonal values are cases where the model used to generate the attack and the model used for evaluation are the same VLM; off-diagonal values are transfer attack results between different VLMs.

As shown, MiniGPT-4 and InstructBLIP exhibit a transfer attack effect on each other’s adversarial images. This result indicates that some perturbation patterns are not tied to one specific model and carry a transfer effect to other VLMs.

However, transfer attacks are generally weaker than white-box attacks, and as with LLaVA the effect varies considerably by model pairing.

Note also that the paper’s transferability table reports the best jailbreak result among the unconstrained setting and the three $\epsilon$ settings.

Comparing the Efficiency of Text and Image Attacks

The paper ran image and text adversarial attacks with the same objective, the same number of iterations, and the same batch size.

In that comparison, the visual attack achieved a lower optimization loss than the text attack. In a text attack, gradients with respect to token embeddings can be computed, but discrete token IDs cannot be updated directly by gradient descent. The AutoPrompt-based attack used in the paper therefore performs gradient-guided candidate token search and evaluation, which in this implementation cost roughly 12 times more computation than the visual attack.

In both the human evaluation and the RealToxicityPrompts evaluation, the visual attack generally showed a stronger effect than the corresponding text attack baseline. It was not, however, consistently superior across every perturbation budget and every metric.

Since the comparison target is a single AutoPrompt-based text attack, it is hard to conclude from these results that visual attacks are stronger than all text jailbreak attacks.

Limitations

First, the main attack assumes a white-box setting with access to model parameters and gradients. This may be a realistic assumption for open-source VLMs, but the same attack is hard to apply directly to black-box models (e.g., ChatGPT, Claude). Transfer-based black-box attacks are also evaluated, but their performance is lower than white-box attacks and varies widely by model pairing.

Second, care is needed in interpreting the metrics. The toxicity detection rate measured on RealToxicityPrompts and the success rate of carrying out harmful instructions are different metrics. An increase in toxic outputs does not mean the model concretely carried out the user’s harmful instruction. Those results must therefore be interpreted separately from the attack success rate in the human evaluation.

Finally, the comparison with text attacks was carried out against a single AutoPrompt-based text attack baseline. Even though the visual attack generally showed a stronger effect than that baseline, it was not consistently superior across every perturbation budget and evaluation metric. It is therefore hard to generalize from this result that visual attacks are stronger than all text jailbreak attacks.

Conclusion

This paper shows that even when safety alignment is applied to the LLM backbone, that alone does not guarantee the adversarial robustness of the whole VLM. An attacker can generate an adversarial image by optimizing only the image pixels with PGD so as to minimize the generation loss of a small harmful target corpus. The resulting adversarial image can act as a universal visual jailbreaker that not only produces specific target sentences but also bypasses the model’s refusal behavior for a variety of harmful text instructions.

The paper shows that in a VLM, the LLM being safety-aligned is separate from the safety of the whole VLM.

Accordingly, evaluating the safety of a VLM requires assessing not only the harmfulness of the text prompt or the LLM’s safety alignment, but also the adversarial robustness of the entire end-to-end pipeline running from the text-generation loss through the visual encoder and projection module down to the input image.