passdrill
AI & LLM Engineering · Fine-tuning & Model Customisation · Card 004/010 easy

A general-purpose language model that previously solved multi-step arithmetic and logic problems reliably is fine-tuned exclusively on a narrow customer-support ticket dataset for several epochs. Afterward, the model handles support tickets well but its arithmetic and logic performance has dropped sharply, even though none of the fine-tuning data was arithmetic-related. What phenomenon does this describe, and what tends to make it worse?

  1. This is overfitting to noise in the customer-support dataset, meaning the model has memorized incorrect labels in that specific dataset; it has nothing to do with the model's previously learned capabilities, which fine-tuning cannot affect
  2. This is a tokenizer mismatch, meaning the fine-tuning process silently changed the model's vocabulary so that arithmetic expressions are now encoded into different tokens than during pretraining
  3. This is catastrophic forgetting: fine-tuning updates the model's weights to optimize performance on the new, narrow task distribution, and in doing so can overwrite representations the model relied on for previously learned capabilities; this effect tends to become more pronounced the further the fine-tuning data diverges from the pretraining distribution and can worsen with more fine-tuning steps or a higher learning rate
  4. This is a known evaluation artifact where the arithmetic benchmark itself becomes unreliable after any fine-tuning run, regardless of what the fine-tuning data contained, so the drop is not a real change in the model's underlying ability
Next card → Shuffle