Min et al. (2022), "Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?," test in-context learning by randomly replacing the labels in few-shot demonstrations with incorrect ones. What did they find, and what does it suggest about why few-shot demonstrations help?
- Replacing the labels with random ones caused accuracy to collapse to chance level on every task tested, confirming that the model learns the exact input-label mapping shown in the demonstrations the way a supervised classifier would
- Replacing the labels with random ones improved accuracy beyond using correct labels, showing that few-shot demonstrations actively mislead the model and should generally be avoided
- The experiment could not be run, because in-context learning requires every demonstration label to be verified against a held-out validation set before each query
- Replacing the demonstrations' labels with random, often-incorrect ones barely hurt accuracy across a range of classification and multiple-choice tasks, suggesting that the correctness of the input-label mapping matters far less than other aspects of the demonstrations, such as the label space, the input distribution, and the overall format
Why D? And why not the others?
Correct answer: D. Replacing the demonstrations' labels with random, often-incorrect ones barely hurt accuracy across a range of classification and multiple-choice tasks, suggesting that the correctness of the input-label mapping matters far less than other aspects of the demonstrations, such as the label space, the input distribution, and the overall format
Min et al. (2022) find that swapping in random, frequently incorrect labels for the demonstrations' true labels causes only a small drop in accuracy, consistently across 12 different models and a range of classification and multi-choice tasks, whereas removing other properties of the demonstrations (such as their label space or input distribution) hurts much more. This suggests in-context learning does not primarily work by the model learning the specific input-to-label mapping shown, the way a supervised classifier fits training pairs, but instead benefits from being shown the format, the space of valid labels, and the distribution of inputs. The second option is wrong because the paper's central, surprising finding is that accuracy did not collapse to chance when labels were randomized; it stayed close to the correct-label condition. The third option is wrong because random labels did not outperform correct labels; the finding is that they are roughly comparable, not that random labels are actively better or that demonstrations mislead the model. The fourth option is nonsensical and describes no real methodological constraint of in-context learning.
Source: Min, Lyu, Holtzman, Artetxe, Lewis, Hajishirzi, Zettlemoyer, "Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?" (EMNLP 2022), arXiv:2202.12837