Catastrophic forgetting

You fine-tune a model on Swedish support answers. It becomes excellent at that — and starts answering worse on everything else: code, English, mathematics, following instructions.

That is called catastrophic forgetting: when the weights are moved towards the new task, what was encoded in the same weights is lost.

The insidious part is that you do not notice it, because you are only measuring on the new task. Your eval shows 0.91 and everything looks fine — until a user asks a question outside the domain.

The rule: always measure two things after fine-tuning — the target task and a general durability suite.

The countermeasures, in order of effect:

The measure	The mechanism	The cost
PEFT (LoRA) with a low rank	the base weights are never touched	none
A low learning rate (1e-5 to 5e-5)	small steps away from the starting point	slower convergence
Fewer epochs (1–3)	less drift	it can underfit
Mixed data (replay)	5–20 % general instruction data in the training mix	you need such data
A KL penalty against the reference model	it limits how far the policy may go	extra computation
Model merging (task arithmetic)	interpolate between the base and the fine-tuned model	it requires evaluating the weighting

The measurement: define a durability suite of 100–200 cases from abilities you want to keep (following instructions, another language, simple mathematics, code snippets). Run it before and after. A deterioration of more than a few percentage points is a deliberate decision that has to be justified — not a surprise.

# Mixed data (replay): keep the general ability during fine-tuning
import random

def build_training_mix(domain_data, general_data, general_share=0.10, seed=0):
    rng = random.Random(seed)
    n_gen = int(len(domain_data) * general_share / (1 - general_share))
    mix = list(domain_data) + rng.sample(general_data, min(n_gen, len(general_data)))
    rng.shuffle(mix)
    return mix

# Measure before and after on BOTH suites
def report(model_before, model_after, target_eval, durability_eval):
    r = {}
    for name, suite in (("target", target_eval), ("durability", durability_eval)):
        b, a = suite(model_before), suite(model_after)
        r[name] = {"before": round(b, 3), "after": round(a, 3), "delta": round(a - b, 3)}
    return r

# {'target':     {'before': 0.62, 'after': 0.91, 'delta': 0.29},
#  'durability': {'before': 0.78, 'after': 0.59, 'delta': -0.19}}   ← this is forgetting

The table above is exactly the report that should accompany every fine-tuned model. Without the «durability» row nobody knows what the adaptation cost.