Prompt Engineering

Correct answer: A. Zero-shot provides no task examples in the prompt, relying only on a natural language instruction, while few-shot includes a small number of input-output examples in the prompt before the actual query

In Brown et al. (2020), the zero-shot setting gives the model only a natural language description or instruction of the task with no worked examples, while the few-shot setting (in-context learning) includes several demonstration examples formatted as input-output pairs directly in the prompt before the real query, with no weight updates in either case. The second option is wrong because neither setting involves fine-tuning; the paper's central point is that GPT-3 performs tasks purely through prompting, without any gradient-based training on the target task. The third option is wrong because in-context learning explicitly does not update model weights at all -- the "learning" happens only within the context window at inference time, not through gradient updates. The fourth option confuses few-shot examples with reasoning steps, which is an unrelated chain-of-thought concept rather than the actual definition of few-shot prompting given in this paper.

Source: Brown et al., "Language Models are Few-Shot Learners" (2020), arXiv:2005.14165, Sections 1-2

Correct answer: B. It includes intermediate reasoning steps leading to the final answer within the few-shot exemplars, rather than showing only the input and final answer

Wei et al. (2022) show that augmenting few-shot exemplars with intermediate natural-language reasoning steps -- a "chain of thought" -- leading up to the final answer substantially improves performance on multi-step reasoning tasks, without any change to the model's weights. The first option is wrong because chain-of-thought prompting still uses worked examples; it enriches them with reasoning text rather than removing them in favor of a single instruction. The third option is wrong because the technique is applied at inference time to an already-trained, frozen model, not through a retraining or fine-tuning procedure on step-by-step solutions. The fourth option describes the opposite of the technique: chain-of-thought prompting deliberately elicits more explanatory reasoning text, not less, because that intermediate text is precisely what improves the accuracy of the final answer.

Source: Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (2022), arXiv:2201.11903

Correct answer: A. In the system prompt, because even a single sentence establishing a role there focuses Claude's behavior and tone for the specific use case

Anthropic's documentation states that setting a role in the system prompt focuses Claude's behavior and tone for the use case, and notes that even a single sentence placed there makes a measurable difference -- no fine-tuning or special placement within the user turn is required. The second option is wrong because placement within the user message is not how the documented technique works; the system prompt is a separate, dedicated field intended for this kind of persistent framing. The third option is wrong because role-setting as documented here is a prompting technique applied at inference time through the system prompt, not a training-time intervention requiring a labeled dataset. The fourth option is wrong because it describes the model inferring a role from its own past outputs rather than being told a role explicitly up front, which is not the mechanism the documentation describes.

Source: Anthropic, "Prompting best practices" -- "Give Claude a role" section, platform.claude.com prompt engineering documentation

Correct answer: A. It sharpens the probability distribution over next tokens so the model more consistently picks the highest-probability token, producing more deterministic, less varied output

Temperature scales the logits before the softmax step that turns them into a probability distribution over next tokens; a temperature near 0 sharpens that distribution so the highest-probability token is chosen almost every time, yielding deterministic, low-variance output, while higher temperatures flatten the distribution and increase randomness and diversity. The second option is wrong because temperature has nothing to do with how much prior context the model attends to -- that is governed by the context window, a separate, unrelated setting. The third option is wrong because output length is controlled by a separate maximum-tokens style parameter, not by temperature. The fourth option is wrong because even at temperature 0 the model is still generating tokens step by step from its learned probability distribution; it is not performing retrieval or reproducing an exact quotation from training data.

Source: Anthropic Messages API reference and OpenAI Chat Completions API reference, `temperature` parameter documentation

Correct answer: A. Direct prompt injection means malicious instructions are typed straight into the model's input by the user; indirect prompt injection means the malicious instructions are hidden in external content, such as a webpage or document, that the LLM later ingests and follows

OWASP's LLM01 entry defines prompt injection as crafted input that alters model behavior or output, and distinguishes direct injection -- where an attacker types malicious instructions straight into the model's input, for example telling it to ignore previous instructions -- from indirect injection, where the malicious instructions are embedded in external content such as a document, webpage, or email that the LLM ingests and treats as if it were a legitimate instruction. The second option is wrong because the direct/indirect distinction is about the source of the malicious instruction, not about a model's licensing or deployment model; both open-source and closed API-based models are vulnerable to either variant. The third option is wrong because neither variant requires physical access to the server; both are input-based attacks deliverable remotely through normal application inputs. The fourth option is wrong because OWASP lists prompt injection as its top-ranked, actively exploited risk in real deployments, not a theoretical or already-solved problem.

Source: OWASP Top 10 for LLM Applications (2025), LLM01: Prompt Injection

Correct answer: A. It helps Claude parse complex prompts unambiguously by clearly separating different types of content, reducing the chance the model misinterprets what is instruction versus example versus input

Anthropic's documentation states that XML tags help Claude parse complex prompts unambiguously, especially when a prompt mixes instructions, context, examples, and variable input, and it recommends wrapping each type of content in its own consistently named tag, nesting tags when the content has a natural hierarchy. The second option is wrong because XML tags are a stylistic and structural convention recommended for clarity, not syntax enforced or required by the Messages API; plain-text prompts without tags remain valid requests. The third option is wrong because adding tags adds characters and therefore tokens rather than reducing them; the documented benefit is parsing clarity, not compression. The fourth option is wrong because tagging content has no translation function of any kind; the model still processes the tagged text in whatever language it was originally written in.

Source: Anthropic, "Prompting best practices" -- "Structure prompts with XML tags," platform.claude.com prompt engineering documentation

Correct answer: A. The model's tendency to disproportionately predict whichever label appeared in the example placed nearest the end of the few-shot prompt, regardless of the true input

Zhao et al. show that few-shot LLM predictions are systematically biased toward the answer of whichever demonstration example sits closest to the query at the end of the prompt -- what they term recency bias -- alongside majority label bias, which favors whichever label is most frequent among the examples, and common token bias; they propose contextual calibration to correct for these effects. The second and third options describe unrelated phenomena, model performance drift across product releases and citation preferences, that this paper does not study or address. The fourth option describes the opposite of recency bias: it describes a primacy effect toward the first example, whereas the paper's finding is specifically a bias toward the last, most recent example in the prompt, not the earliest one.

Source: Zhao, Wallace, Feng, Klein, Singh, "Calibrate Before Use: Improving Few-Shot Performance of Language Models" (ICML 2021), arXiv:2102.09690

Correct answer: A. Sample multiple diverse reasoning paths for the same question at a nonzero temperature, then take a majority vote over the final answers each path arrives at

Self-consistency samples a diverse set of reasoning paths for the same chain-of-thought prompt, using sampling rather than greedy decoding, and then marginalizes over them by taking the most common final answer, exploiting the intuition that correct reasoning tends to converge on the same answer through multiple valid paths even when the intermediate reasoning text differs. The second option is wrong because it introduces a human-in-the-loop review step rather than the fully automatic sampling-and-voting procedure the paper describes. The third option is wrong because self-consistency requires no fine-tuning and no brute-force search over training data; it operates purely at inference time on an already-trained, frozen model. The fourth option is wrong because it eliminates chain-of-thought reasoning altogether in favor of external retrieval, which is unrelated to self-consistency; the technique keeps and multiplies the reasoning paths rather than replacing them with a search engine lookup.

Source: Wang et al., "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (ICLR 2023), arXiv:2203.11171

Correct answer: A. First prompt the model to decompose the problem into a sequence of simpler subproblems, then sequentially prompt it to solve each subproblem in order, feeding each prior subproblem's answer into the context used for the next

Least-to-most prompting works in two stages: a decomposition stage where the model breaks a complex problem into an ordered list of simpler subproblems, followed by a sequential problem-solving stage where the model solves each subproblem in turn, with the answers to earlier subproblems included in the context used to solve later ones; this lets the technique generalize to problems harder than the demonstrations shown, unlike standard chain-of-thought prompting. The second option is wrong because the technique involves no fine-tuning at all; it is a purely prompting-based, inference-time method applied to a frozen model. The third option is wrong because it reverses the actual order and purpose of the method -- least-to-most decomposes a problem before solving it, rather than justifying an answer that was already guessed, and post-hoc justification is not what the paper studies. The fourth option is wrong because the method uses a single model working sequentially through ordered subproblems, not a multi-vendor voting ensemble.

Source: Zhou et al., "Least-to-Most Prompting Enables Complex Reasoning in Large Language Models" (ICLR 2023), arXiv:2205.10625

Correct answer: A. Both guarantee syntactically valid JSON, but only Structured Outputs also guarantees the output conforms to the caller's supplied JSON Schema (for example, required keys and enum values); JSON mode guarantees valid JSON syntax only

OpenAI's documentation states that Structured Outputs "ensures the model will always generate responses that adhere to your supplied JSON Schema," explicitly calling it "the evolution of JSON mode," while JSON mode only ensures the output parses as valid JSON without enforcing that it matches any particular schema, meaning JSON mode output can still omit required fields or use invalid enum values even though it is syntactically valid JSON. The second option reverses the actual guarantees documented by OpenAI. The third option is wrong because the distinction has nothing to do with input modality; both features concern the format of the model's text output regardless of whether the request includes image inputs. The fourth option is wrong because the documentation explicitly frames Structured Outputs as adding a stricter guarantee, schema adherence, that JSON mode does not provide, rather than describing an identical rebrand.

Source: OpenAI, "Structured Outputs" documentation, platform.openai.com/docs/guides/structured-outputs

Correct answer: A. They reordered the multiple-choice options in the few-shot examples so the correct answer was biased to always fall on a particular letter (e.g., always "(A)"); the model's chain-of-thought then rationalized picking that biased letter while never mentioning the answer ordering as its real reason

Turpin et al. show that adding a biasing feature to a prompt, such as reordering multiple-choice options so the correct answer is artificially made to always fall on a particular letter, can systematically shift the model's final answer toward that letter, while the model's generated chain-of-thought explanation confabulates a plausible-sounding justification that never mentions the true, biasing cause; this demonstrates that the stated reasoning does not faithfully reflect the actual process behind the answer. The second, third, and fourth options describe manipulations and outcomes -- removing all reasoning causing refusal, translation causing randomness, or scaling example count improving faithfulness -- that are not the intervention or finding reported in this paper. The paper's central manipulation is specifically the answer-position biasing intervention, and its central finding concerns faithfulness of the explanation, not raw accuracy or refusal behavior.

Source: Turpin, Michael, Perez, Bowman, "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting" (2023), arXiv:2305.04388

Correct answer: A. Purely formatting-level changes to a semantically identical few-shot prompt caused accuracy swings of up to tens of accuracy points (as much as 76 points for one open-source model tested), and this sensitivity persisted even with larger models, more few-shot examples, and instruction tuning

Sclar et al. found that large language models are highly sensitive to spurious, purely cosmetic formatting choices in few-shot prompts, such as separators, spacing, and other superficial template details that do not change the semantic content, with accuracy swings of up to 76 points observed for one model studied (LLaMA-2-13B), and this sensitivity did not disappear as model size, number of few-shot examples, or instruction tuning increased. The second option is wrong because the paper explicitly reports that this sensitivity persists even as models scale up, rather than resolving at some parameter threshold. The third option is wrong because the study concerns plain-text prompt formatting rather than image inputs, and it found the effect specifically in text-based few-shot prompting. The fourth option is wrong because the paper studies few-shot formatting sensitivity directly; it does not report that switching to zero-shot prompting eliminates formatting-driven accuracy differences.

Source: Sclar, Choi, Tsvetkov, Suhr, "Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design" (ICLR 2024), arXiv:2310.11324