Project F: fine-tune and evaluate a language model

Project F: take an open model, adapt it to a task you care about, and show with numbers that it got better — without getting worse at anything else.

The deliverables:

The part	The requirement
The dataset	≥ 500 pairs, cleaned (duplicates, eval leakage), balanced, documented
The eval	≥ 50 cases measuring the target task, built before the training
The durability suite	≥ 50 cases for general abilities that are to be preserved
The baselines	the base model as it is, plus a prompted variant (few-shot)
The training	LoRA/QLoRA, the configuration logged, ≥ 2 seeds
The report	before and after on both suites, with the spread; the limitations

The requirement that makes the project honest: the eval suite is built before the training. Otherwise it is unconsciously adapted to what the model happened to become good at.

The result table you are to produce:

The variant	The target eval	Durability	Trainable par.	GPU time
base, zero-shot	0.41 ± 0.00	0.78 ± 0.00	—	—
base, few-shot	0.58 ± 0.01	0.78 ± 0.00	—	—
LoRA r=8	0.79 ± 0.02	0.75 ± 0.01	15 M	25 min
LoRA r=32	0.82 ± 0.02	0.71 ± 0.02	61 M	31 min

Read it: r=32 gives +3 pp on the target but −4 pp on the durability. Is that the right choice? It depends on the use — and that decision is the point of the project, not the highest figure.

Common mistakes:

1. The few-shot baseline is skipped — and then nobody knows whether the fine-tuning was even needed.
2. Only one seed.
3. The durability suite is missing.
4. The chat template differs between the training and the evaluation (a silent loss of quality).
5. The eval cases written afterwards.

The timetable (about 5 h of active time): 1.5 h the dataset · 1 h the eval suites · 1 h the training (a sweep) · 1 h the evaluation · 0.5 h the report.