The Transformer decoder (Vaswani et al., 2017) contains an "encoder-decoder attention" sublayer in addition to its own masked self-attention sublayer. Where do the queries, keys, and values for this encoder-decoder attention sublayer come from?
- The queries come from the decoder's own previous sublayer, while the keys and values come from the output of the encoder stack, letting every decoder position attend over the entire input sequence
- The queries, keys, and values all come from the encoder's final layer, and the decoder only reads the resulting attention output without contributing any of its own vectors
- The queries and keys come from the encoder output while the values come from the decoder's own previous sublayer, reversing the usual roles of queries and keys
- The queries, keys, and values all come from the decoder's own previous sublayer, identical to the masked self-attention sublayer that precedes it
Why A? And why not the others?
Correct answer: A. The queries come from the decoder's own previous sublayer, while the keys and values come from the output of the encoder stack, letting every decoder position attend over the entire input sequence
In the encoder-decoder attention sublayer, Vaswani et al. take the queries from the output of the decoder's own preceding sublayer, while the keys and values come from the output of the encoder stack; this lets every position in the decoder attend over all positions in the input sequence, mirroring the query-key-value structure used in earlier sequence-to-sequence attention mechanisms. The option in which queries, keys, and values all come from the encoder is wrong because the decoder must still contribute its own queries so that each decoder position can ask a different question of the encoder output. The option that swaps queries/keys and values between encoder and decoder is wrong because Vaswani et al. specifically place the keys together with the values from the same source (the encoder) rather than splitting queries and keys across two different sources. The option describing queries, keys, and values all coming from the decoder is wrong because that describes the masked self-attention sublayer earlier in the decoder block, not the separate encoder-decoder attention sublayer that follows it.
Source: Vaswani et al., "Attention Is All You Need" (2017), arXiv:1706.03762, Section 3.2.3 (Applications of Attention in our Model)