Paper: Jailbreak in Pieces: Compositional Adversarial Attacks on Multi-Modal Language Models
Authors: Erfan Shayegani, Yue Dong & Nael Abu-Ghazaleh
Venue: ICLR 2024
URL: Jailbreak in Pieces: Compositional Adversarial Attacks on Multi-Modal Language Models
Jailbreak in Pieces
Introduction
Prior work on visual adversarial attacks manipulated image pixels to elicit harmful responses from the attached LLM. Because such methods assume a white-box setting in which the attacker can access the model’s parameters and gradients, the same approach was hard to apply directly in settings without access to the model internals (black-box models).
This paper, by contrast, does not directly optimize the LLM’s output probabilities but instead targets the embedding produced by the vision encoder. It splits a harmful request across text and an image, so that when the two inputs are combined the model is steered into generating a harmful response.
The paper therefore addresses the following research questions.
If a harmful request is split across text and image input, can safety alignment be bypassed? And can the harmful meaning contained in the image be hidden without direct access to the LLM?
The paper’s main contributions can be summarized as follows.
- It introduces a compositional jailbreak attack method in which the meanings of a text instruction and an image combine.
- It presents a way to hide harmful meaning in an adversarial image using the vision encoder’s embedding, without access to the LLM.
Attack Method
Jailbreak in Pieces
The paper assumes the attacker can access a vision encoder such as CLIP and compute gradients with respect to the image input. It does not require access to the LLM’s parameters or the linear layer. The attack is therefore black-box with respect to the LLM, but not a fully black-box attack against the whole VLM.
The attacker sends the model an adversarial image combined with a text prompt. The text prompt describes the task to perform, but is phrased so that the object of that task is taken from the image.
Compositional Attack
Instead of delivering the entire harmful request as text, this paper’s attack splits the task to perform and the object of that task across different inputs.
The text describes what the model should do, and the image describes the object or situation that the instruction refers to. Interpreting the two inputs together, the model pulls from the image the content not stated explicitly in the text.
In this construction, a single adversarial image can be used with several text instructions. Conversely, different adversarial images can be attached to a single text instruction. Compositionality, as the paper uses the term, means precisely this ability to recombine and reuse images and text.
Constructing the Malicious Trigger
The attacker must first decide what meaning the adversarial image should carry. The paper calls this goal the malicious trigger and constructs the target embedding in the following four ways.
| Trigger type | How the target embedding is constructed |
|---|---|
| Textual Trigger | Harmful text is fed into CLIP’s text encoder. |
| OCR Textual Trigger | An image with harmful text written on it is fed into the vision encoder. |
| Visual Trigger | An image depicting a harmful object or scene is fed into the vision encoder. |
| Combined Trigger | An image containing both text and a visual object is fed into the vision encoder. |
The four methods differ in how the embedding that serves as the optimization target is constructed. The textual trigger is also ultimately used to generate an adversarial image.
Moreover, feeding in a harmful image or phrase as-is would let a human or an image filter see the content. To avoid this, the paper generates images that have an embedding close to the original trigger while not revealing that content on the surface.
Optimization Objective
The attacker optimizes the image pixels so that the adversarial image’s embedding approaches the malicious trigger’s embedding. This can be expressed as follows. \(\hat{x}_{\mathrm{adv}} = \underset{x_{\mathrm{adv}}\in\mathcal{B}}{\arg\min} \mathcal{L}_{2} \left( H_{\mathrm{harm}}, \mathcal{I}_{\phi}(x_{\mathrm{adv}}) \right)\)
Here $x_{\mathrm{adv}}$ is the image being optimized and $H_{\mathrm{harm}}$ is the fixed target embedding. $\mathcal{I}{\phi}$ is the vision encoder, $\mathcal{L}{2}$ is the $L_2$ distance between the two embeddings, and $\mathcal{B}$ is the constraint applied to the image input.
What the attack directly minimizes is the embedding distance between the adversarial image and the target. Because of this, the optimization never needs to compute gradients through the LLM’s response.
Optimization Method
First, the target trigger is fed into the encoder to compute $H_{\mathrm{harm}}$, which is then fixed. The embedding of the initial image is computed next, and the image pixels are updated repeatedly so that the $L_2$ loss between the two embeddings decreases. This process backpropagates through the vision encoder down to the image pixels.
Experimental Results
The paper evaluates the attack against LLaVA and LLaMA-Adapter V2. The CLIP encoders used for the attack are ViT-L/14-336 and ViT-B/32 respectively, and Adam’s learning rate was set to $0.1$.
Evaluation Datasets and Metrics
The paper uses 8 scenarios in total, related to sexual content, hate, violence, self-harm, and harassment. In each scenario, 8 adversarial images are generated using the 4 trigger strategies and evaluated together with 2 generic prompts.
For human evaluation, three raters judge harmfulness based on the model’s output and the corresponding scenario, without seeing the input image or the trigger used in optimization.
Success is judged by whether the model actually carried out the harmful request. A response that merely describes the object or warns about the danger is not counted as success, while providing the requested harmful content counts as success even if a warning is included.
Performance under Human Evaluation
The average attack success rate for each trigger is as follows.
The textual trigger almost never succeeded on either model, whereas attacks using a target embedding obtained from an image showed high success rates. The combined trigger is highest on average, but it is not always better than the other triggers in every scenario.
The important point here is that what mattered greatly was not whether the image contains text, but which modality’s encoder the target embedding was obtained through.
Performance under Automatic Evaluation
The paper also ran automatic evaluation on LLaMA-Adapter V2’s outputs using the Perspective API and a toxicity classifier. Toxicity scores were low for the textual trigger, but higher for the visual and combined triggers.
These results show a trend similar to the human evaluation. Note, however, that the toxicity score and the success rate of carrying out harmful instructions are different metrics. The mere fact that a highly toxic sentence was generated should therefore not be read as the model having accurately performed the requested task.
Conclusion
This paper shows that in evaluating a VLM’s safety alignment, examining text and image separately is not enough: the meaning produced when the two inputs combine must also be considered. If the object of a harmful request is delivered separately through an image, an attack can succeed that would never be visible in the text alone.
The attacker does not modify the model parameters and optimizes only the image pixels so that the embedding distance at the vision encoder decreases. The image generated this way can be reused with several generic textual instructions and serve as a component of a compositional jailbreak.