02 Training and Adaptation 3 min read 657 words

Fine-Tuning Strategies

Fine-tuning adapts a pretrained model to specific tasks, domains, or styles. In 2025, fine-tuning is less about "teaching facts" and more about "teaching format and behavior."

fine-tuningtrainingproductioncore

01fine-tune

When to Fine-Tune

Before fine-tuning, ask: Can this be solved with Prompt Engineering or RAG?

Requirement	Better Solution	Why
New Facts / Knowledge	RAG	LLMs are bad at memorizing facts from FT; RAG is easier to update.
Specific Output Format	Fine-Tuning	Teaches the model to reliably output JSON/XML without complex prompting.
Tone / Persona	Fine-Tuning	Much more consistent than system prompts.
Latency Reduction	Fine-Tuning	Reduces the need for long few-shot prompts.
Private Domain Language	Continued Pretraining	Teaches specialized vocabulary (medical, legal, custom code).

02fine-tuning sft

Supervised Fine-Tuning (SFT)

The first step after pretraining. The model is trained on (Prompt, Response) pairs.

2.1

The 2025 Quality Hierarchy

In 2025, 1,000 "Perfect" examples beat 1,000,000 noisy examples.

Golden Sets: Hand-curated by domain experts (PhD level for technical tasks).
Negative Constraint Training: Including examples of what the model should not do (e.g., "Don't apologize," "Don't mention you are an AI").

03domain adaptation

Continued Pretraining (Domain Adaptation)

Also known as "Second-stage Pretraining."

How: Train on raw text from a specific domain (e.g., all SEC filings for a finance model).
Objective: Learn the statistical distribution of the domain language.
Nuance: Requires a much lower learning rate (~1/10th of original) to prevent "catastrophic forgetting."

04vs. full-parameter

PEFT vs. Full-Parameter

Feature	Full-Parameter FT	PEFT (LoRA, QLoRA)
GPU VRAM	Very High (Model Size * 4-12)	Low (Model Size * 1.5)
Speed	Base	2x-3x Faster
Risk	High (Catastrophic Forgetting)	Low
Deployment	One model per task	One base model + multiple adapters
2025 Verdict	Reserved for foundation training	The Production Standard

05dec nuances

Hyperparameter Tuning (Dec 2025 Nuances)

5.1

1. Learning Rate (LR)

SFT: 1e-5 to 5e-5 is standard.
Too high: Model "collapses" and starts repeating or speaking gibberish.

5.2

2. Rank (r) for LoRA

In 2025, we use higher ranks (r=64 to r=256) for complex reasoning tasks.
Lower ranks (r=8) are only for simple style/tone changes.

5.3

3. Packaged Training (Packing)

To maximize throughput, we "pack" multiple short examples into a single 4k or 8k sequence, separated by EOS tokens.

Challenge: Self-attention might leak across examples.
2025 Solution: FlashAttention with block-masking to prevent cross-example attention.

06questions

Interview Questions

Q: Why use Continued Pretraining instead of just putting domain data in the SFT set?

Strong answer: SFT is "expensive" in terms of data creation—you need prompt/answer pairs. Continued Pretraining allows you to leverage massive amounts of raw, unlabeled domain text to teach the model's inner representations the specialized vocabulary and style. Once the model "speaks the language," you use a small SFT set to teach it the "tasks" (e.g., classification, summarization) in that language.

Q: How do you prevent a model from "unlearning" general capabilities during fine-tuning?

Strong answer: This is "Catastrophic Forgetting." Two main mitigations:

Rehearsal: Mix in 5-10% of the original pretraining data into your fine-tuning set.
PEFT (LoRA): Since we only train a small percentage of weights, the original "knowledge" remains frozen in the base model weights, significantly reducing the risk of forgetting.

07references

References

Hu et al. "LoRA: Low-Rank Adaptation of Large Language Models" (2021)
Ouyang et al. "Training language models to follow instructions" (InstructGPT, 2022)
Dettmers et al. "QLoRA: Efficient Finetuning of Quantized LLMs" (2023)

Next: LoRA, QLoRA, and PEFT

• summary · added by this rebuild

Key takeaways

Fine-tuning teaches form, RAG teaches facts

The decision table sends new knowledge to RAG because it is easier to update, and reserves fine-tuning for reliable output formats, persona consistency and shorter prompts.

A thousand clean examples win

The quality hierarchy favours expert-curated golden sets, plus negative constraint examples that show the model explicitly what it should not do.

PEFT is the production default

LoRA and QLoRA need about 1.5 times model size in VRAM against 4 to 12 for full fine-tuning, train two to three times faster, and ship as one base plus swappable adapters.

LoRA rank should track task difficulty

Rank 8 only carries style and tone; complex reasoning wants 64 to 256, with SFT learning rates held between 1e-5 and 5e-5 before the model collapses into repetition.