Zou et al. (2023), "Universal and Transferable Adversarial Attacks on Aligned Language Models," introduce the Greedy Coordinate Gradient (GCG) method for finding a short string that, appended to a harmful request, causes safety-trained models to comply. How does GCG find this adversarial suffix, and what does "transferable" mean in the paper's results?
- GCG works by manually testing suffixes proposed by a human red team, ranking them only by how grammatically fluent they read, with no use of gradients or automated search at all
- GCG uses gradient information from one or more open-weight models to greedily search for, and iteratively replace, individual tokens in a candidate suffix so as to increase the likelihood that the target model begins its response by complying with the harmful request; "transferable" describes the finding that a suffix optimized against open-weight models such as Vicuna also induces objectionable output when tested against unrelated closed models it was never optimized against
- GCG requires direct write access to the target model's weights during the attack itself, so it cannot be used against a closed model served only through an API, and "transferable" refers only to porting the attack's code between programming languages
- GCG produces a suffix that is unique to a single specific harmful request and a single specific model, and "transferable" describes only how the resulting text can be copy-pasted between chat interfaces of the same model
Why B? And why not the others?
Correct answer: B. GCG uses gradient information from one or more open-weight models to greedily search for, and iteratively replace, individual tokens in a candidate suffix so as to increase the likelihood that the target model begins its response by complying with the harmful request; "transferable" describes the finding that a suffix optimized against open-weight models such as Vicuna also induces objectionable output when tested against unrelated closed models it was never optimized against
GCG is an optimization-based attack: it uses gradients computed on one or more open-weight models to identify, at each position in a candidate suffix, which token substitutions would most increase the probability that the target model's response begins by complying with the harmful request, then greedily applies the best substitution found and repeats across many iterations until an effective suffix emerges. The paper's transferability result is that a suffix optimized this way against accessible open-weight models such as Vicuna-7B and 13B still induces objectionable output when later tested against entirely different, closed models it was never optimized against directly, including public chat interfaces the attacker never had gradient access to. The option describing manual human testing ranked by fluency is wrong because GCG is an automated, gradient-guided search, and the resulting suffixes are typically not fluent at all. The option claiming direct weight access to the target model is required is wrong because the transfer result specifically demonstrates success against models the attacker never had such access to. The option restricting transferability to copy-pasting within one model's interfaces is wrong because the paper's finding is that the suffix generalizes across different underlying models, not just across interfaces of the same model.
Source: Zou, Wang, Carlini, Nasr, Kolter & Fredrikson, 'Universal and Transferable Adversarial Attacks on Aligned Language Models' (arXiv:2307.15043, 2023)