Many LLM provider APIs offer a "streaming" mode for text generation, typically implemented over Server-Sent Events (SSE). What does enabling streaming change about how a client receives the model's output?
- The client receives the complete response in a single payload, but compressed with gzip to reduce total bandwidth compared to a non-streaming request
- The model generates its answer in a single internal pass regardless of the setting; streaming only changes how the response is logged on the provider's servers, not how the client receives it
- The client receives the response as a sequence of incremental chunks over an open connection as tokens are generated, rather than waiting for generation to finish before anything is returned
- The client must poll a separate status endpoint at fixed intervals to check whether generation has completed, since no data is returned until the full response is ready
Why C? And why not the others?
Correct answer: C. The client receives the response as a sequence of incremental chunks over an open connection as tokens are generated, rather than waiting for generation to finish before anything is returned
Streaming delivers partial output incrementally, as a sequence of small chunks sent as events over an open HTTP connection while the model is still generating, letting a client start displaying tokens (for example, in a chat UI) well before the full response completes, which lowers perceived latency for long responses. Gzip compression of a single complete payload is unrelated to streaming -- it is an orthogonal transport-level optimization that applies equally to non-streaming responses and does not change how many payloads are sent. Streaming very much changes what the client itself receives over the wire, not just how the provider logs the response server-side; the client's connection stays open and receives multiple discrete events in order. Polling a separate status endpoint describes an asynchronous batch-style pattern, which is a different integration model from token-by-token streaming and is typically used for long-running bulk jobs rather than conversational responses.
Source: OpenAI, API reference on streaming responses, https://platform.openai.com/docs/api-reference/streaming; Anthropic, Messages API streaming documentation, https://platform.claude.com/docs/en/build-with-claude/streaming