Fine-tuning

What is fine-tuning?

Fine-tuning is the practice of taking a model that has already been pre-trained on general data and training it further on a smaller, labeled dataset so it performs one task, or speaks in one voice, more reliably than the base model did. The weights change, so the behavior persists without being restated in every prompt.

The scale gap is the point. Pre-training consumes trillions of tokens and industrial compute budgets; a support-team fine-tune typically runs on a few hundred to a few thousand curated examples and finishes in hours on rented hardware.

How fine-tuning works

Fine-tuning runs as five stages: base selection, dataset curation, the training run, evaluation, and versioned deployment.

Base selection sets the ceiling. A large language model that already handles your languages and reasoning depth will absorb a fine-tune cleanly, while a weaker base learns your format and still reasons badly underneath it.

Curation absorbs most of the effort. Each row pairs an input with the exact output you want returned, so the dataset encodes your policy decisions as firmly as your tone. Teams usually start from resolved ticket transcripts and rewrite the agent replies into the form they wish had been sent.

The training run then adjusts weights by penalizing the distance between the model's output and the labeled target over repeated passes. Behavior learned this way persists across every future request, which is what separates it from few-shot learning, where the examples live in the prompt and vanish when the request ends, and from prompt engineering, which edits instructions and leaves the weights untouched.

Evaluation scores a held-out set the model never trained on, and deployment pins a version so a bad fine-tune can be rolled back.

Types of fine-tuning, with examples

• Full fine-tuning: Every weight in the model is updated, giving maximum control at the highest compute and storage cost per variant you keep.

• Parameter-efficient fine-tuning: Small adapter matrices such as LoRA train while the base stays frozen, for example one adapter per brand served from a shared base.

• Instruction tuning: The model trains on prompt-and-response pairs so it follows house format, for example always returning a refund decision alongside the policy clause behind it.

• Preference optimization: Labeled pairs teach the model which of two candidate answers is better, the mechanism behind direct preference optimization.

• Continued pre-training: Unlabeled domain text such as claims notes or device manuals shifts the model's vocabulary before any task-specific tuning begins.

Fine-tuning vs prompt engineering vs RAG vs pre-training

These four get compared as if a team had to pick one, and that framing causes most of the wasted spend in this area. Pre-training builds general language ability from raw corpora at a scale almost no support team will ever fund. Prompt engineering changes behavior by editing text in the context window, in minutes. Retrieval-augmented generation supplies facts at answer time from a source you control and can cite. Fine-tuning changes the weights themselves, so form, tone, and task behavior hold without instruction.

Start with prompt engineering, because it is reversible in an afternoon. Add retrieval-augmented generation when the wrong answers are factual and your policies change weekly. Reach for fine-tuning when the model already knows the facts and keeps expressing them in the wrong shape.

Why fine-tuning matters for customer experience

When a model behaves inconsistently and nobody can tune it, teams compensate in the prompt. The system prompt grows: a tone rule here, an exception there, a paragraph nobody remembers adding. Past a certain length those instructions start contradicting each other, the model follows whichever came last, and every request pays for the extra tokens in latency and cost.

Fine-tuning moves the stable parts of that prompt into the weights. Formatting, refusal behavior, escalation phrasing, and voice stop being re-argued on every turn, and the prompt shrinks back to the case-specific facts.

The tradeoff is reversibility. A prompt line is edited in a minute; a learned behavior takes a new dataset, a training run, and a fresh evaluation before it shifts, so anything volatile belongs outside the weights.

How is fine-tuning measured?

No standards body sets a target score that a support team is expected to hit for a fine-tuned model. Public leaderboards such as MMLU do exist, and they measure general capability on academic multiple-choice tasks, so their taxonomy is not your taxonomy and a score there says little about whether the model handles your refund policy.

What transfers is method. Build a held-out set of real tickets the model never trained on, score the base model and the tuned model against the same rubric, and compare answers pairwise, which gives reviewers a judgment they can make consistently. Keep a regression set of cases the model previously got wrong and rerun it on every release.

The NIST AI Risk Management Framework organizes this under four functions, govern, map, measure, and manage, and its measure function is where the evaluation set, the rerun cadence, and the named reviewer belong.

How AI agents change fine-tuning

An AI support agent does more than write text: it decides whether to call a tool, which arguments to pass, when to refuse, and when to hand off. Those decisions are structured outputs, so training targets shift from well-written paragraphs to correctly shaped tool calls, escalation choices, and refusals under uncertainty. A model that writes beautifully and formats a refund call wrongly still fails the job.

The data source shifts too. Agent traces, each holding the question, the retrieved context, the tool calls, and the outcome, become the raw material, and the label is whether the resolution held without the customer coming back.

The consequence lands downstream. Every fine-tune moves the model's calibration, so the thresholds deciding when the agent answers and when it escalates have to be re-checked afterward; the confidence threshold tuning playbook covers that rerun.

What to look for in a fine-tuning setup

Judge a setup on the axes that decide whether you can still operate it next year.

Coverage: confirm behavior genuinely differs across products, languages, and support tiers, because one adapter serving everything is far cheaper to maintain than five. Integration surface: adapters should swap per brand without redeploying the base, and stay callable from one runtime. Governance: the dataset needs a named owner and a version pinned to the model version, because a training set is a policy document that gets read as evidence when an answer is challenged.

Two frameworks bite here. SOC 2 Type II applies to whoever stores your labeled transcripts, which are real customer conversations. GDPR bites harder, since an erasure request cannot be satisfied by deleting a row from trained weights, so personal data has to be stripped before any example is used.

The constraint teams underestimate is base model deprecation: when a provider retires the checkpoint you tuned against, the adapter does not port, and only the dataset survives to train its replacement.

Fine-tuning and AI grounding

Fine-tuning changes how a model answers; AI grounding changes what it is permitted to answer from. A tuned model with no grounded source still produces confident specifics when the fact is missing, because nothing in the weights separates a remembered policy from an invented one. Grounding supplies the passage and the citation, and the tuned behavior decides how that passage becomes a reply. Teams working out which lever to pull usually start with the difference between grounding and retrieval, since grounding names the goal and retrieval is one method of reaching it.

What does fine-tuning mean in plain terms?

Think of a base model as someone who has read the entire library and never worked a day in your building. Fine-tuning is their first month on the floor: they already know the words, and now they learn how your team answers, what your team never promises, and where the line for a manager sits.

Without it, you repeat those house rules at the start of every single conversation, and you find out which one got dropped only when a customer reads the answer.

The tradeoff is that lessons learned this way stick. Change the refund window next quarter and the model keeps answering with the old one until someone gathers new examples and trains again, which is why the facts that move belong somewhere you can edit in an afternoon.

Common fine-tuning mistakes

Four patterns account for most of the damage.

Using a fine-tune to fix a knowledge gap is the first. Training teaches shape and behavior, and any fact absorbed into weights arrives without a source, cannot be cited, and stays wrong until the next training run. Facts belong in a retrievable store.

Training on raw historical transcripts is the second. Every past mistake in that queue, the wrong refund call, the misquoted policy, the escalation that never happened, becomes a labeled example teaching the model to repeat it. Transcripts are raw material and need editing before they become training data.

Evaluating on data the model trained on is the third. The scores look excellent and describe memorization, and the failure appears in production on phrasing the set never contained.

Skipping the safety layer is the fourth. A tuned model is more consistent and just as capable of acting outside policy, which is why guardrails for support automation sit outside the model and get enforced at runtime.