Building with LLM APIs

Why B? And why not the others?

Correct answer: B. The response contains a structured description of the function name and the arguments the model wants to pass; the calling application must run the actual function itself and send the result back in a follow-up request

This is correct because standard function/tool calling returns a structured block naming the tool and the arguments the model wants to pass; the model itself has no way to execute arbitrary code or reach external systems, so the calling application is responsible for actually running the corresponding function and returning its result in a subsequent request so the model can incorporate it. The option describing vendor-side execution confuses this with built-in "server tools" (such as a hosted web-search tool) that some vendors run on their own infrastructure -- that is the exception, not the general mechanic of user-defined function calling. The option about the model running the function internally is wrong because a language model's weights encode learned associations, not a general-purpose interpreter capable of arbitrary code execution. The option requiring the function's output before the first request is backwards: the whole point of the feature is that the model requests the call first, before any result exists.

Source: Anthropic, 'Tool use with Claude' overview, https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview; OpenAI, 'Function calling' guide, https://platform.openai.com/docs/guides/function-calling

Why C? And why not the others?

Correct answer: C. The client receives the response as a sequence of incremental chunks over an open connection as tokens are generated, rather than waiting for generation to finish before anything is returned

Streaming delivers partial output incrementally, as a sequence of small chunks sent as events over an open HTTP connection while the model is still generating, letting a client start displaying tokens (for example, in a chat UI) well before the full response completes, which lowers perceived latency for long responses. Gzip compression of a single complete payload is unrelated to streaming -- it is an orthogonal transport-level optimization that applies equally to non-streaming responses and does not change how many payloads are sent. Streaming very much changes what the client itself receives over the wire, not just how the provider logs the response server-side; the client's connection stays open and receives multiple discrete events in order. Polling a separate status endpoint describes an asynchronous batch-style pattern, which is a different integration model from token-by-token streaming and is typically used for long-running bulk jobs rather than conversational responses.

Source: OpenAI, API reference on streaming responses, https://platform.openai.com/docs/api-reference/streaming; Anthropic, Messages API streaming documentation, https://platform.claude.com/docs/en/build-with-claude/streaming

Why A? And why not the others?

Correct answer: A. The combined total of the input tokens sent in the request (system prompt, conversation history, and any injected documents) plus the output tokens the model generates in response

The context window is a hard ceiling on the total number of tokens the model can attend to in one call, and that total includes everything fed in as input (system prompt, prior conversation turns, retrieved documents, tool definitions) plus every token the model produces as output; if the combined total would exceed the limit, the request is rejected or there is no room left for the model to generate. Treating only the latest message as counted is wrong because unless a client deliberately truncates or summarizes prior turns itself before sending the request, full conversation history sent as input still counts toward the same shared budget -- there is no automatic summarization performed by the API. Treating only output tokens as counted is wrong because input processing consumes the same limited context, not a separate unlimited channel. Confusing this with a per-minute request cap conflates the context window with rate limiting, a completely separate constraint governing how many calls can be made over time, not how much content fits within one call.

Source: Anthropic, 'Models overview' context window documentation, https://platform.claude.com/docs/en/about-claude/models/overview; OpenAI, 'Models' documentation on context window, https://platform.openai.com/docs/models

Why D? And why not the others?

Correct answer: D. Back off and retry after a delay that increases with each subsequent failure (exponential backoff), typically with some added random jitter, rather than retrying immediately or at a fixed short interval

A 429 status code signals that the client has exceeded a rate or usage limit (such as requests per minute or tokens per minute), and the standard remediation across REST-style LLM provider APIs is exponential backoff: wait progressively longer between retries as failures continue, with random jitter added so that many concurrent clients don't all retry at the exact same moment and cause a new burst of failures. Retrying immediately and aggressively is the opposite of what is recommended, since it worsens the very condition causing the 429s rather than relieving it. A 429 on one endpoint reflects that specific endpoint's or account's usage limit, not a platform-wide outage, so switching to an unrelated endpoint does not address the underlying limit at all. Reducing `max_tokens` addresses a different failure mode entirely -- a request that would generate too much output or exceed a context-length limit -- not a rate-limit error, which is about request or token volume over time rather than the size of any single request.

Source: OpenAI, 'Rate limits' guide, https://platform.openai.com/docs/guides/rate-limits; Anthropic, 'Rate limits' documentation, https://platform.claude.com/docs/en/api/rate-limits

Why B? And why not the others?

Correct answer: B. The embeddings endpoint takes text as input and returns a fixed-length numeric vector representing that text's meaning, rather than generating new natural-language text

An embeddings endpoint converts input text into a fixed-length numeric vector that captures its semantic content, positioning similar meanings closer together in vector space; this is used for tasks like semantic search, clustering, and retrieval rather than producing readable prose, which is fundamentally different from a generation endpoint that autoregressively produces new natural-language tokens. Describing it as merely a "faster" generation endpoint is wrong because it does not generate text at all -- it returns numbers, not sentences. Restricting embeddings to images only is wrong, since provider embeddings endpoints primarily operate on text (some also support images), while the generation endpoint's core case is likewise text, so the two are not split along an image-versus-text line. Requiring fine-tuning first is wrong because base embeddings models are available for immediate use via the API just like base generation models, with fine-tuning being an optional enhancement for either endpoint type rather than a prerequisite for either one.

Source: OpenAI, 'Embeddings' guide, https://platform.openai.com/docs/guides/embeddings

Why A? And why not the others?

Correct answer: A. It tells the API to stop generating further tokens as soon as any one of a specified list of strings appears in the output, and to exclude that string from the returned text

Stop sequences let a caller specify one or more strings that, if generated, cause the API to immediately halt generation at that point and return the output produced so far, with the matched stop string itself typically omitted from the returned text; this is commonly used to cut a response off at a natural boundary, such as before a model would start role-playing the next turn of a conversation. A wall-clock timeout is a distinct, separate control unrelated to matching specific text content, and would cut a response off regardless of what was actually generated. A blanket prohibition on ever producing certain words describes content filtering or constrained decoding, not what stop sequences do, since stop sequences only trigger termination once a match completes rather than preventing the string from ever being produced. Pausing for manual approval mid-stream describes an entirely different, human-in-the-loop interaction pattern that generation APIs do not implement via this parameter.

Source: OpenAI, Chat Completions API reference, `stop` parameter, https://platform.openai.com/docs/api-reference/chat/create; Anthropic, Messages API reference, `stop_sequences` parameter, https://platform.claude.com/docs/en/api/messages

Why C? And why not the others?

Correct answer: C. By setting `tool_choice` to `{"type": "tool", "name": "get_weather"}`, which forces the model to call that specific named tool rather than deciding on its own or calling a different one

Anthropic's Messages API exposes a `tool_choice` parameter with four possible values: `auto` (the default, letting the model decide whether and which tool to call), `any` (forcing some tool call but not a particular one), `none` (preventing tool use), and `tool` -- and setting `tool_choice` to `{"type": "tool", "name": "get_weather"}` specifically forces that named tool to be called on this turn, which is exactly the guarantee the developer wants. Removing the other tool from the array would only prevent it from being offered at all, and does not actually force a call, since the model could still decline to call the only remaining tool under the default `auto` behavior. The `any` value forces some tool call, but not a specific one -- it does not default to "the first tool listed"; the model still chooses among whichever tools are available to it. Relying only on a system-prompt instruction is a weaker, unenforced nudge that Anthropic's own documentation contrasts with the `tool_choice` mechanism, which structurally guarantees the behavior rather than merely encouraging it through prompting.

Source: Anthropic, 'Define tools' -- Forcing tool use, https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools

Why D? And why not the others?

Correct answer: D. With Chat Completions, the client must append prior turns into the message array and resend the full history each call, while the Responses API can instead reference a prior turn via a `previous_response_id` parameter so the server carries the context forward

OpenAI's documentation describes Chat Completions as requiring the client to manually reconstruct conversation state by including the model's previous outputs as part of the input and appending that to each new request's message array, whereas the Responses API offers a `previous_response_id` parameter as a way to chain a new request to an earlier response so the server-side context is carried forward without the client re-sending everything itself. The first option reverses which API requires manual resending -- it is Chat Completions, not the Responses API, that lacks this shortcut. The option claiming both APIs are purely manual with no alternative ignores the Responses API's `previous_response_id` mechanism entirely. The option claiming the Responses API cannot maintain multi-turn context at all is wrong because chaining via `previous_response_id`, along with the related Conversations API for longer-running threads, is specifically built for exactly that purpose.

Source: OpenAI, 'Conversation state' guide, https://platform.openai.com/docs/guides/conversation-state

Why A? And why not the others?

Correct answer: A. Having the tool accept a unique idempotency key per logical operation, so the backend can recognize a retried call with the same key and return the original result instead of processing the charge a second time

The generally correct fix is at the tool and backend layer, not the model layer: attaching a unique idempotency key to each logical operation lets the backend detect that a retried request represents the same intended operation and short-circuit to returning the original result rather than executing the side effect again, which is the standard pattern for exactly this kind of network-timeout-then-retry scenario in payment and other side-effecting APIs, and is the recommended orchestration-layer practice for agent tool calls that can be retried. Increasing `max_tokens` only affects how much text the model can generate and has no bearing on whether a backend call is executed twice. Lowering temperature to 0 only makes the model's chosen arguments more deterministic across similar prompts; it does nothing to prevent the exact same request from actually being re-sent and re-executed after a timeout, since the retry happens regardless of how deterministic the arguments were. Blanket-disabling repeat calls to any tool would also break legitimate cases where a tool is meant to be called multiple times with different arguments in the same conversation, making it too blunt a fix for this specific failure mode.

Source: OpenAI, 'In production' guide for Agentic Commerce (idempotency keys for retried tool/side-effect calls), https://developers.openai.com/commerce/guides/production

Why B? And why not the others?

Correct answer: B. They report, for each generated token, the log probability the model assigned to it (and often to alternative candidate tokens), and are typically used to gauge the model's confidence or build classifiers from token likelihoods

Log probabilities express, on a log scale, how likely the model considered each token it actually output (and optionally some number of alternative top candidate tokens at that position), which is useful for gauging the model's confidence in its own output, flagging likely hallucinations where confidence is unusually low, or building lightweight classifiers on top of a generation task by comparing the relative log probabilities of a small set of candidate answers. They have nothing to do with latency, which providers report separately, if at all, as timing metadata rather than as a probability value. They also are not a per-token cost breakdown; billing is typically reported as aggregate input and output token counts rather than a probability-shaped field attached to each token. And they do not identify which fine-tuned checkpoint generated a token -- that information, if available at all, would be reported as separate model or version metadata on the response, not encoded in a probability value.

Source: OpenAI, Chat Completions API reference, `logprobs`/`top_logprobs` parameters, https://platform.openai.com/docs/api-reference/chat/create

Why C? And why not the others?

Correct answer: C. `top_p` restricts sampling to the smallest set of most-probable tokens whose cumulative probability reaches the threshold `p`, so the number of candidates varies step to step, while `top_k` always keeps a fixed number of the highest-probability tokens regardless of how the probability mass is distributed

Nucleus sampling (`top_p`) builds a candidate pool by adding tokens in order of probability until their cumulative probability reaches the threshold `p`, so a peaked distribution yields a small pool and a flatter distribution yields a larger one -- the pool size adapts to the model's confidence at each generation step. `top_k` sampling instead always keeps exactly the same fixed number, `k`, of the highest-probability tokens as candidates regardless of how concentrated or spread out the probability mass is, which can include unlikely tokens when the true distribution is sharply peaked, or exclude reasonable ones when it is flat. The two are not the same mechanism expressed in different units, since one produces a variable-size candidate pool and the other a fixed-size one. The option describing `top_k` as the variable, confidence-dependent one and `top_p` as fixed reverses their actual behavior. Both parameters shape the candidate pool independently of `temperature` and take effect at any temperature setting, not only at exactly 0, where sampling becomes fully deterministic and such cutoffs become moot only in the trivial sense that a single token already dominates the distribution.

Source: OpenAI, API reference `top_p` parameter documentation, https://platform.openai.com/docs/api-reference/chat/create; Holtzman et al. (2020), 'The Curious Case of Neural Text Degeneration' (arXiv:1904.09751), introducing nucleus sampling

Why D? And why not the others?

Correct answer: D. The Batch API offers roughly a 50% cost discount compared to the synchronous API, but processes the submitted batch of requests asynchronously with results typically available within 24 hours rather than immediately, and draws from a separate rate-limit pool

OpenAI documents the Batch API as accepting a file of many requests, each with its own identifier for matching results back to inputs, that is processed asynchronously against a separate rate-limit pool, at roughly a 50% cost discount relative to the equivalent synchronous requests, with completion typically well within the fixed 24-hour completion window rather than returned immediately -- a tradeoff of latency for cost that fits exactly the non-latency-sensitive bulk classification use case described. The claim of same price but a guaranteed 60-second turnaround is wrong on both counts: the price is discounted relative to the synchronous endpoint, and results are not guaranteed within any such short window. The claim that it is entirely free with a mandatory week-long wait invents figures the documentation does not state; the actual discount is roughly half price, not zero, and the completion window is 24 hours, not 7 days. The claim of a one-request-per-batch limit is backwards: a single batch file is specifically built to bundle a large number of requests, documented up to tens of thousands, into one submission, which is the entire point of the feature for a use case like this one.

Source: OpenAI, Batch API guide, https://developers.openai.com/api/docs/guides/batch