Ten fine-tuning mistakes I see students make (and made myself)
DEV Community

Ten fine-tuning mistakes I see students make (and made myself)

I built FineTune Studio after making most of these mistakes on my own runs. They are not exotic. They are the difference between "I fine-tuned a model" on a resume and "I fine-tuned a model and here is what improved by how much".

The Mistakes

  • No baseline. Not running the base model on the same evaluation set first. Without it, you cannot know whether the fine-tune did anything. Base-versus-tuned is the whole point.
  • Training on unvalidated data. Malformed records, duplicates, and template mismatches. Run the pre-training checklist first.
  • Leaking eval into training. Splitting after deduplication is not optional.
  • Chasing training loss. It goes down whether or not the model is getting better. Watch eval loss, then read outputs.
  • Too many epochs on too little data. A few hundred examples for ten epochs is memorisation with a progress bar.
  • Choosing the biggest model that fits. A 1-3B model fine-tuned well beats a 7B model fine-tuned badly, and ships on hardware you have.
  • Fine-tuning for facts. Knowledge belongs in retrieval; see RAG vs fine-tuning.
  • Ignoring the chat template. Training with one format and serving with another produces a model that seems to have forgotten everything.
  • Not saving checkpoints. The best model was at step 600; you only kept step 1200.
  • No deployment plan. An adapter on a laptop is not a result anyone else can use. Decide how it will be served before you start.

The Habit That Prevents Most of Them

Write the evaluation before the training script: the held-out set, the rubric, and the baseline numbers. Everything else becomes an experiment against a fixed target instead of a hope. That habit, more than any hyperparameter, is what I try to teach in campus sessions on fine-tuning. Make these mistakes once, on a small model, on a weekend. Then never again.

About Pranjul Rathour

Pranjul Rathour - GenAI engineer, Kanpur. Speaking at a MeetKats event. Taking questions during a session. At VSICS, Kanpur. At an Integral Startup Foundation hackathon. Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG, fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at student hackathons across India.

Updated 2026-09-06. Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pra*************@gmail.com · Invite me to your campus Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.