Khattab et al. (2023), "DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines," argue that hand-written free-form prompt templates discovered by trial and error make LLM pipelines brittle and hard to reuse across models. What does DSPy have the programmer do instead, and what does the framework itself handle automatically?
- The programmer writes the exact final prompt wording as before, and DSPy's only contribution is translating that wording into several different natural languages automatically
- The programmer must manually rewrite every prompt for every new base model DSPy is pointed at, since the framework provides no automatic prompt generation or tuning of its own
- The programmer specifies only the desired output token count, and DSPy pads or truncates the model's natural response to match that fixed length regardless of prompt wording
- The programmer declares a "signature" describing, in a structured and model-agnostic way, what inputs a step needs and what outputs it should produce, and composes these signatures into modules forming a pipeline; DSPy's own compiler then automatically generates and tunes the actual natural-language prompt text, and can select or generate few-shot demonstrations, needed to make each declared step work
Why D? And why not the others?
Correct answer: D. The programmer declares a "signature" describing, in a structured and model-agnostic way, what inputs a step needs and what outputs it should produce, and composes these signatures into modules forming a pipeline; DSPy's own compiler then automatically generates and tunes the actual natural-language prompt text, and can select or generate few-shot demonstrations, needed to make each declared step work
DSPy replaces hand-crafted free-form prompt strings with a declarative "signature," a structured, model-agnostic description of what inputs a pipeline step consumes and what outputs it should produce, and lets the programmer compose several such signatures into modules that form a full pipeline. Rather than the programmer discovering and hard-coding the exact wording that makes a given base model behave correctly, DSPy's own compiler takes over that job: it automatically generates and tunes the natural-language prompt text for each declared step, and can also select or generate the few-shot demonstrations used within that prompt, so the same declared pipeline can be recompiled for a different base model without the programmer rewriting prompt wording by hand. The option describing automatic translation into other natural languages is wrong because DSPy's compilation targets prompt effectiveness for a given model and task, not multilingual translation of a fixed wording. The option claiming DSPy provides no automatic prompt generation is wrong because automatic prompt generation and tuning by the compiler is the paper's central contribution. The option describing padding or truncating output to a fixed token count is wrong because DSPy's declared signatures describe input/output structure and intent, not a fixed output length enforced independently of prompt wording.
Source: Khattab, Singhvi, Maheshwari, Zhang, Santhanam, Vardhamanan, Haq, Sharma, Joshi, Moazam et al., 'DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines' (arXiv:2310.03714, 2023)