Fine-Tuning Decisions and LoRA Concepts

Mateo in Bogotá leads support for a bike shop. The assistant told a customer that returns are accepted for thirty days; the current policy says fourteen. "Can we fine-tune the model on the right answer?" he asks. (An illustrative case.) We could, and the answer would be right until the policy changes again. Then it would be wrong in a place nobody can read or diff, because a weight is not a document with a version. The failure was stale data, and fine-tuning is the wrong tool for it.

This page teaches you to tell that kind of problem from the kind fine-tuning does help, and what careful work comes first if it is the second kind: a baseline, a clean dataset with rights and splits, and a working picture of what a LoRA adapter is. Fine-tuning can improve consistent task behaviour, but it is costly and often worse than improving data, prompts, retrieval or output validation. Everything here runs in numpy on synthetic data: no GPU, model download or real training, and the toy training shows mechanics, not how a real model behaves. The next page, PEFT Experiments and Model Rights, builds on this code.

What you will learn

• How to separate style and task-pattern problems from missing or changing knowledge, and to choose between prompting, retrieval, structured output, ordinary code and fine-tuning with a decision table and a decision record.
• How to save a baseline before training and to build a small synthetic dataset that carries its own rights fields.
• How to split by group and validate examples for schema, duplicates, conflicting targets, leakage, secrets and personal data.
• What supervised fine-tuning, parameter-efficient fine-tuning and LoRA's low-rank update are, with a numpy layer you can check by hand.
• How to read parameter counts, rank, scaling, overfitting and drift on a toy task, and how to produce a dry-run report that invents nothing.

What is new, and what is reused

No existing file changes. The new subpackage evidencedesk/finetune/ has decide (symptom routes, Baseline, readiness questions, DecisionRecord), data (the example format, validator, group split), flawed (a training file with planted flaws, written on demand), lora, toy (training demo and checks), budget (a labelled hypothetical calculator) and dryrun (AdapterConfig and the report). Data is in data/finetune/; tests are test_finetune_*.py. Reused: the scanners of Safe Failure and Security Regression Tests and Prompt Injection and Access Boundaries, overlap from Context Quality, Budgets, and Reproducibility, and the baseline file of Rubrics, Regression, and Release Gates. The habits of Evaluation Datasets and Deterministic Tests (versioned data, a reason per case) apply again in small.

Identify problems fine-tuning can and cannot solve

Weights hold behaviour: how to phrase, which format, which label, how to act on a familiar pattern. They are a poor place for facts that change (a policy, a price, a recall list) because they cannot be edited, cited or rolled back one fact at a time. So first name the cause of a failure, then pick the fix. decide.py keeps the table as data: eight symptoms (the scenario file scenarios.jsonl), each with a cause, the first fix to try, and whether fine-tuning is relevant.

from evidencedesk.finetune.decide import load_scenarios, route_for

for s in load_scenarios():
    r = route_for(s["symptom"])
    print(f'{s["id"]} {s["symptom"]:<19}{r.fine_tune:<10}{r.first_fix}')

s01 stale_fact         no        update the data, filter superseded versions, retrieve
s02 exact_term_miss    no        keyword search beside embeddings, then measure
s03 format_failure     after     schema, stop-reason check, bounded retry
s04 no_evidence_answer no        retrieval threshold, margin and a refusal path
s05 style_inconsistent candidate a clearer prompt and examples, measured against a rubric
s06 stable_labelling   candidate a prompt with examples, then a small classifier as a rival
s07 arithmetic         no        ordinary code behind a tool
s08 injection          no        access boundaries and tool gates in code

Only two symptoms are candidates, and even they say "a prompt first, measured". Format failures are marked after: use the contract from Reliable Output Contracts first, and consider tuning only if a measured gap remains. The rest are not about behaviour at all. Look back at what the Kestrel assistant got wrong so far: the superseded returns policy was a data problem, fixed by a status filter (RAG Reliability and Evaluation Data); invalid JSON needed a contract; the saved baseline failures rc03 (an exact identifier) and rc06, rc08, rc13 (paraphrases) were retrieval misses, which Hybrid Retrieval and Error Analysis and Reranking and Retrieval Evaluation measured and moved; lg03 expects a refusal in Bangla and needs an abstention rule. None is a weights problem; a tuned model would have hidden them behind fluent text.

Compare prompting, RAG, and fine-tuning

A decision table by what each method changes. "Cost to change" is relative and unmeasured; it describes the shape of the work, not a price.

The methods combine: a fine-tuned model still needs retrieval for current facts and the same output checks. Try the cheapest reversible change first, each measured on the same evaluation set.

Provider note (as of 2026-10-12): Microsoft's Foundry overview says fine-tuning does not replace retrieval for current information or application-level safety controls, lists style, format, tool use and shorter prompts as uses, and says benefits are not guaranteed. OpenAI's optimisation guide advises building evaluations and trying prompt engineering first, and says its fine-tuning platform is no longer open to new users; the Gemini API documentation says no model there currently supports fine-tuning (it remains on the enterprise platform). Check the current documentation before planning around a service.

Define a baseline before training

"Better after tuning" needs a "before". A Baseline keeps four things: scores, latency, the configuration that produced them, and the ids of the failing examples. It loads from the evaluation file saved by Rubrics, Regression, and Release Gates and lists what it still lacks. Latency was never measured there, so it is None and reported as a gap; a guess would be worse. Beside it, eight readiness questions (a missing answer counts as no).

from evidencedesk.finetune.decide import DATA, QUESTIONS, Baseline, DecisionRecord, readiness

base = Baseline.from_eval(DATA / "baseline_v2.json", "kestrel-v2")
print(base.name, "pass_rate", base.metrics["pass_rate"], "recall_at_k", base.metrics["recall_at_k"], "failures", list(base.failures))
print("gaps in the baseline:", base.problems())

answers = {"baseline": True, "prompt_tried": False, "first_fix_tried": False, "behaviour_problem": True,
           "enough_examples": True, "data_rights": True, "stable_task": True, "rollback": False}
unmet = readiness(answers)
for q in unmet:
    print("unmet:", q)

kestrel-v2 pass_rate 0.848 recall_at_k 0.882 failures ['lg03', 'rc03', 'rc06', 'rc08', 'rc13']
gaps in the baseline: ['latency not measured']
unmet: A clearer prompt and examples were tried and measured.
unmet: The first fix for the cause (data, retrieval, schema, code) was tried and measured.
unmet: A gate decides keep or roll back, and the old route stays available.

The questions come out as a to-do list, not a verdict: "allowed to try" never means "will help". Now the decision record, a short document you can read in six months. It lists the options considered, the choice, the evidence and the risks. Its checker refuses fine_tune while a baseline is missing or a readiness question is open.

record = DecisionRecord(
    id="dec-001", date="2026-10-13", problem="Replies differ in tone and length across agents (scenario s05).",
    symptom="style_inconsistent", options=["prompt", "structured_output", "fine_tune"], decision="prompt",
    evidence=["baseline kestrel-v2: pass_rate 0.848", "prompt with examples not yet measured against a style rubric"],
    risks=["a longer prompt costs tokens on every call"], review_when="after the prompt is scored on the style rubric",
    baseline="kestrel-v2", unmet=unmet)
print(record.to_markdown())
print("problems:", record.problems())
print("if we wrote decision='fine_tune':", DecisionRecord(**{**record.__dict__, "decision": "fine_tune"}).problems())

# Decision dec-001 (2026-10-13)
Problem: Replies differ in tone and length across agents (scenario s05).
Symptom: style_inconsistent
Options considered: prompt, structured_output, fine_tune
Decision: prompt
Evidence:
- baseline kestrel-v2: pass_rate 0.848
- prompt with examples not yet measured against a style rubric
Risks:
- a longer prompt costs tokens on every call
Baseline: kestrel-v2
Review when: after the prompt is scored on the style rubric
problems: []
if we wrote decision='fine_tune': ['fine_tune chosen with a baseline missing or readiness questions unmet']

The record picks "prompt": the style problem is real, but nobody has yet measured a prompt with examples against a style rubric.

Create a small synthetic training dataset

The task we would tune is a style: one short answer sentence, then Next step: and one action. The targets contain no numbers, because a number copied into a target becomes a stale fact the model will state with confidence. Facts stay in retrieval; the adapter would learn only the shape. The file has 30 hand-written chats in ten topic groups (kestrel_sft.jsonl), all labelled synthetic. The first record, one JSON object per line in the file:

from evidencedesk.finetune.data import load_examples

e = load_examples()[0]
print(e.id, e.group, e.source, e.licence, "|", e.provenance)
for turn in e.messages:
    print(f"{turn.role:<10}{turn.content}")

ex-001 returns synthetic internal-synthetic | hand-written by the course author for this tutorial, 2026-10-13
system    You are Kestrel Cycles support. Reply in the house style: one short answer sentence, then 'Next step:' and one action.
user      What is the time limit for sending a bike back?
assistant Returns follow the current returns policy page. Next step: open the returns page and start a return request.

Besides messages (an optional system turn, one user turn, one assistant turn), each record has group (examples that share a topic), and three rights fields: source, licence and provenance (who wrote it, how, when). Thirty rows teach mechanics, not a real model; how many a real task needs is found by measuring. to_prompt_completion gives the prompt-plus-completion shape.

Tool note (as of 2026-10-12): The TRL SFTTrainer documentation lists four dataset shapes (plain text, messages, prompt and completion, and a conversational prompt-completion) and applies the chat template to conversational data automatically. It also says adapters usually train with a higher learning rate, about 1e-4, than full training. Azure's guide uses separate training and validation JSONL files. Formats differ by tool: check the current documentation.

Separate training, validation, and test data

Three roles: training data changes the weights; validation data guides your choices (rank, number of steps, when to stop); test data is touched once, at the end, so its score is not tuned. Split by group, not by row. Three phrasings of the same return question are one example in three costumes; if two land on opposite sides of the split, validation measures memory. assign_splits moves whole groups with a seeded shuffle, so the same seed gives the same answer.

from evidencedesk.finetune.data import CLEAN, assign_splits, cross_split_leaks, load_examples, split_ids

examples = load_examples(CLEAN)
groups = assign_splits(examples, seed=7)
ids = split_ids(examples, groups)
for name in ("train", "val", "test"):
    print(f"{name:<5}{len(ids[name]):>3} examples, groups {sorted(g for g, s in groups.items() if s == name)}")
print("same input in two splits:", cross_split_leaks(examples, groups))

train 18 examples, groups ['assembly', 'giftcards', 'returns', 'service', 'shipping', 'warranty']
val    6 examples, groups ['privacy', 'sizing']
test   6 examples, groups ['battery', 'recalls']
same input in two splits: []

Tool note (as of 2026-10-12): scikit-learn's pitfalls page defines data leakage as using information that would not be available at prediction time and says to split into train and test subsets first, before any preprocessing. Azure's fine-tuning guide says to keep validation and final test examples out of the training set. Check the current documentation.

Validate training examples

validate reads a file line by line and never raises on bad input: it reports. The checks: JSON and schema (including the chat shape); duplicate ids; exact and near-duplicate inputs (word overlap at least 0.8); label consistency, meaning the same input must not have two different answers; leakage against every question in the evaluation files; secrets and personal data (the Beginner scanners); injection-like text (a poisoning hint); numbers in targets; and the rights fields. Run it on the clean file and on a second file with ten planted flaws, which write_flawed writes on demand so the kit's data folder never holds a secret-shaped string.

from pathlib import Path

from evidencedesk.finetune.data import CLEAN, eval_texts, validate
from evidencedesk.finetune.flawed import write_flawed

FLAWED = write_flawed(Path("flawed.jsonl"))  # ten planted flaws, written here and not stored in data/
evals = eval_texts()
print(len(evals), "evaluation questions loaded to check for leakage")
print(validate(CLEAN, evals).text())
print(validate(FLAWED, evals).text())

22 evaluation questions loaded to check for leakage
30 rows, 0 errors, 0 warnings
11 rows, 11 errors, 2 warnings
  line  2 ex-001  error   duplicate_example same as ex-001
  line  2 ex-001  error   duplicate_id
  line  3 ex-f03  error   conflicting_target ex-001 answers the same input differently
  line  4 ex-f04  error   eval_leak input matches an evaluation question
  line  5 ex-f05  error   personal_data email
  line  6 ex-f06  warning number_in_target a number in a target can go stale
  line  6 ex-f06  error   secret key_shape
  line  7 ex-f07  error   injection_like override
  line  8 ex-f08  warning number_in_target a number in a target can go stale
  line  9 ex-f09  error   licence_not_allowed unknown
  line  9 ex-f09  error   source_not_allowed scraped
  line 10 ex-f10  error   invalid_schema record: roles must be [system,] user, assistant; got ['user']
  line 11 ex-f11  error   invalid_schema provenance: String should have at least 8 characters

Read the flawed report. Line 2 repeats an id and an example. Line 3 gives the same question a different answer than line 1, which teaches inconsistency. Line 4 copies an evaluation question (eval_leak), visible only because we passed the evaluation questions. Secret and personal-data findings print the kind, never the value. A number in a target is a warning to read, not a defect.

A clean report is not proof of good data: the scanners match shapes, so a name or street address in free text passes, and a paraphrase of a held-out question passes below the overlap threshold. Whether the data teaches the right behaviour is shown only by evaluation.

Understand supervised fine-tuning at a high level

Supervised fine-tuning (SFT) shows the model a prompt and a target answer and nudges the weights so the target becomes more likely. The loss is the average of -log p over the target tokens, where p is the probability the model gave each one; many setups count only the answer tokens, not the prompt. Validation loss on held-out examples shows whether the change generalises. (Model basics: How AI Works: LLMs, Tokens and Limitations.) Only the loss needs arithmetic:

import numpy as np

p_answer = np.array([0.50, 0.25, 0.80])  # invented: the probability a model gave each ANSWER token
per_token = -np.log(p_answer)
print("per-token loss", np.round(per_token, 3), "-> mean loss", round(float(per_token.mean()), 3))

per-token loss [0.693 1.386 0.223] -> mean loss 0.768

(The probabilities are invented.) A confident right token costs little; a doubtful one costs more. But a falling loss measures fit to these examples. Whether replies got better is a question for the same holdout, scored as in the baseline, with failures read one by one.

Explain parameter-efficient fine-tuning, and LoRA's low-rank updates

Full fine-tuning updates every weight, and the optimizer keeps extra numbers for each. Parameter-efficient fine-tuning (PEFT) trains a small set of extra parameters and keeps the base model frozen, so the saved result is a small adapter instead of a full copy. LoRA, the common method, replaces an update of a weight matrix W (d_out by d_in) with a product of two thin matrices: W' = W + (alpha / r) * B * A, where A is r by d_in, B is d_out by r, and the rank r is small. Only A and B train. The product has rank at most r, so the update can only move in a few directions; that is the trade-off, fewer trainable parameters in exchange for less room to change. This is the whole layer:

# file: kit/evidencedesk/finetune/lora.py
"""The LoRA idea in numpy: a frozen weight W plus a low-rank update (alpha / r) * B @ A. Mechanics only; no model, no torch."""
from __future__ import annotations

from dataclasses import dataclass

import numpy as np


def full_params(d_out: int, d_in: int) -> int:
    return d_out * d_in


def lora_params(d_out: int, d_in: int, r: int) -> int:
    """A is r x d_in and B is d_out x r."""
    return r * (d_in + d_out)


@dataclass
class LoraLayer:
    """y = x W^T + scaling * (x A^T) B^T. W is frozen; A and B are the only trainable arrays."""
    W: np.ndarray  # (d_out, d_in), never updated
    A: np.ndarray  # (r, d_in)
    B: np.ndarray  # (d_out, r)
    alpha: float

    @classmethod
    def new(cls, W: np.ndarray, r: int, alpha: float, seed: int = 0) -> LoraLayer:
        """B starts at zero, so the adapted layer equals the base layer before any training."""
        if not 0 < r <= min(W.shape):
            raise ValueError(f"rank must be between 1 and {min(W.shape)}")
        a = np.random.default_rng(seed).normal(0.0, 1.0 / np.sqrt(W.shape[1]), size=(r, W.shape[1]))
        return cls(W, a, np.zeros((W.shape[0], r)), alpha)

    @property
    def r(self) -> int:
        return self.A.shape[0]

    @property
    def scaling(self) -> float:
        return self.alpha / self.r

    def delta(self) -> np.ndarray:
        return self.scaling * (self.B @ self.A)

    def forward(self, x: np.ndarray) -> np.ndarray:
        return x @ self.W.T + self.scaling * ((x @ self.A.T) @ self.B.T)

    def merged(self) -> np.ndarray:
        """One ordinary weight matrix with the adapter folded in."""
        return self.W + self.delta()

    def grads(self, x: np.ndarray, dy: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
        """(dA, dB) for a loss whose gradient with respect to the output is dy. W gets no gradient."""
        return self.scaling * (self.B.T @ dy.T @ x), self.scaling * (dy.T @ (x @ self.A.T))

    def trainable(self) -> int:
        return self.A.size + self.B.size

    def total(self) -> int:
        return self.W.size + self.trainable()

Rules to notice. B starts at zero, so a new adapter changes nothing until training moves it. scaling = alpha / r multiplies the update. grads gives gradients for A and B only; W gets none. The tests check grads against finite differences and check the arithmetic below by hand.

import numpy as np

from evidencedesk.finetune.lora import LoraLayer, full_params, lora_params

layer = LoraLayer(np.eye(2), np.array([[1.0, 2.0]]), np.array([[3.0], [4.0]]), alpha=2.0)  # r = 1, so scaling = 2
x = np.array([[1.0, 1.0]])
print("delta   ", layer.delta().tolist())
print("unmerged", layer.forward(x).tolist(), "| merged", (x @ layer.merged().T).tolist())
for r in (4, 8, 16, 64):
    n = lora_params(4096, 4096, r)
    print(f"hypothetical 4096 x 4096 layer, r={r:<2}: {n:>9,} adapter parameters of {full_params(4096, 4096):,} ({n / full_params(4096, 4096):.2%})")

delta    [[6.0, 12.0], [8.0, 16.0]]
unmerged [[19.0, 25.0]] | merged [[19.0, 25.0]]
hypothetical 4096 x 4096 layer, r=4 :    32,768 adapter parameters of 16,777,216 (0.20%)
hypothetical 4096 x 4096 layer, r=8 :    65,536 adapter parameters of 16,777,216 (0.39%)
hypothetical 4096 x 4096 layer, r=16:   131,072 adapter parameters of 16,777,216 (0.78%)
hypothetical 4096 x 4096 layer, r=64:   524,288 adapter parameters of 16,777,216 (3.12%)

By hand: B A = [[3],[4]] · [[1,2]] = [[3,6],[4,8]], scaled by 2 gives [[6,12],[8,16]]. For x = [1, 1], x Wᵀ = [1, 1] and the adapter term is 2 · (x Aᵀ = 3) · [3, 4] = [18, 24], so [19, 25]. The merged matrix W + delta gives the same answer: merging is plain addition. The table's numbers are pure arithmetic for a hypothetical layer shape, r (d_in + d_out) against d_in · d_out; real counts need the shapes of the layers you target.

Tool note (as of 2026-10-12): The PEFT documentation describes LoRA as decomposing a large matrix into two smaller low-rank matrices while the original weight stays frozen, and says a higher rank means more trainable parameters and more capacity. Its quicktour shows an adapter file of about 6.3 MB beside about 700 MB for the full model it was made from. Check the current documentation; sizes depend on the model, rank and layers.

What changing rank and alpha does, on a toy task

A toy layer (16 by 16) must learn a new behaviour: the true change has rank 2, we built it so, and the 20 training targets are noisy. We train full weights and LoRA at four ranks on five different toy tasks and average. train and val are errors on training and held-out inputs of the new behaviour; old is the error on inputs the new behaviour should not touch, where the base layer is exactly right. The flags use example thresholds (validation more than twice training; old error above 0.25).

from evidencedesk.finetune.toy import average, compare, make_task, table, train_lora

print(table(compare()))
print()
print(table(average([[train_lora(t, 4, a) for a in (1, 4, 16)] for t in map(make_task, range(5))])))

run              trainable   train     val     old  flags
full                   256   0.165   0.211   0.058
lora r=1 a=2            32   0.479   0.444   0.170
lora r=2 a=4            64   0.227   0.108   0.228
lora r=4 a=8           128   0.194   0.114   0.198
lora r=8 a=16          256   0.155   0.166   0.269  drift

run              trainable   train     val     old  flags
lora r=4 a=1           128   0.567   0.604   0.033
lora r=4 a=4           128   0.219   0.124   0.120
lora r=4 a=16          128   0.182   0.141   0.328  drift

Read it as mechanics only. (1) Rank 2 has a quarter of the trainable parameters and about half the validation error of full training: the constraint matches the true change and ignores noise. We built that in; a real change need not be low-rank. (2) Rank 1 cannot represent the change and underfits: too little capacity. (3) Rank 8 has as many parameters as the full layer and starts to behave like it. (4) In the old column the adapters drifted more than full training, which is the opposite of the comforting story that LoRA does not forget. Random A rows reach into directions the new behaviour does not use. One toy says nothing about a real model; the lesson is to measure drift, not assume it. (5) In the second table, only alpha changes, so the update is scaled by 0.25, 1 or 4 at the same learning rate and number of steps: small alpha learns slowly and drifts least, large alpha learns faster and drifts more. Real alpha and learning-rate interplay is for experiments to show.

Overfitting and forgetting checks

One task, rank 8, trained longer and longer:

from evidencedesk.finetune.toy import curve, make_task

for steps, r in curve(make_task(0), 8, 16, (25, 100, 400, 1600, 3200)):
    print(f"{steps:>5} steps  train {r.train:.3f}  val {r.val:.3f}  old {r.old:.3f}")

   25 steps  train 0.702  val 1.214  old 0.106
  100 steps  train 0.298  val 0.735  old 0.228
  400 steps  train 0.137  val 0.212  old 0.312
 1600 steps  train 0.090  val 0.378  old 1.017
 3200 steps  train 0.077  val 0.482  old 1.702

Training error keeps falling. Validation error bottoms out near 400 steps and then rises: the model fits the noise in 20 examples. The old-behaviour error grows throughout. Three habits follow: watch the training-validation gap (the Azure SFT guidance says to inspect failing validation examples when training loss falls and validation loss rises), choose the stopping point on validation and never on test, and keep a regression set of behaviour that must not change and run it before and after. The before-and-after comparison on a real adapter is the next page.

Estimate cost, time and memory, labelled as hypothetical

A back-of-envelope calculator helps with ratios and impossible plans, and misleads when quoted as fact. budget.py counts weights, gradients and optimizer state (two numbers per trained parameter for Adam-style optimizers) and ignores activations. Every rate is a field of Hypothetical.

from evidencedesk.finetune import budget

print(budget.summary(total_params=134_515_008, trainable=1_000_000, train_tokens=5_000_000, epochs=3, saved_tokens=300))

HYPOTHETICAL inputs, not measurements
  memory, full training : 1.88 GB (weights + gradients + optimizer state)
  memory, adapter       : 0.29 GB (frozen weights + adapter state)
  training time         : 2.08 h at 2000 tokens/s
  training cost         : 2.08 units at 1 per hour
  break-even            : 3,472 requests at 300 prompt tokens saved each

The total, 134,515,008, is the parameter count the Hub API reports for one small model's card; the 1,000,000 trainable parameters, the byte sizes, 2,000 tokens per second, the price and the 300 saved tokens are invented. The card's 723.56 MB figure is for loading that model, not training. We state no training time or memory for a laptop CPU: measure on your machine. The break-even line is the useful part: shorter prompts repay a run only after that many requests.

A dry-run report that invents nothing

Where torch, transformers and peft are not installed (not in this course's environment), a training plan is a report. AdapterConfig holds the fields of a LoRA configuration under the names the PEFT library uses (r, lora_alpha, target_modules, lora_dropout, bias, task_type) and prints them as keyword arguments. dry_run checks the config, validates and splits the data, and says what it cannot know.

PYTHONPATH=kit python -m evidencedesk.finetune.dryrun

DRY RUN - nothing was trained, no model was loaded
python 3.12
packages: torch=missing, transformers=missing, peft=missing
config problems: none
adapter: base=a-small-base-model-id r=8 alpha=16 scaling=2 dropout=0.05 modules=('q_proj', 'v_proj') lr=0.0001 epochs=3
data: 30 rows, 0 errors -> ok
split by group: train=18, val=6, test=6
training text: about 1993 tokens (4 characters per token rule, not a tokenizer)
trainable parameters: unknown until a model is loaded (needs its layer names and shapes)
memory, time, cost: not estimated (no measured throughput; see budget.summary for a labelled hypothetical)
would run: NO - torch is not installed; transformers is not installed; peft is not installed
note: availability of any free notebook GPU is not guaranteed; the core lab does not need one

Trainable parameters are "unknown" because they depend on the model's layer names and shapes, which a dry run has not loaded. The package line depends on the machine. Bad data would show BLOCKED. The modules q_proj and v_proj are placeholders: layer names differ between models and we have verified none (VERIFY); the next page prints a real model's module names first.

Tool note (as of 2026-10-12): In the PEFT documentation target_modules takes a list of names (matched exactly or by suffix), a regular expression string, or 'all-linear'; lora_dropout is the dropout probability of the adapter layers; with bias set to all or lora_only, disabling the adapter no longer returns the base model's output. Check the current documentation.

Data rights, as a concept

Training bakes data into a model you may ship; if you had no right to the data, deleting a file does not fix it. So every record carries source, licence and provenance, and the validator refuses a source or licence the project has not allowed. The rights of the base model and of your own adapter are the subject of the next page.

Provider note (as of 2026-10-12): Hugging Face documents model cards as describing intended uses, limits, training information and the datasets used, with the licence as a metadata field; dataset cards carry licence and language too. The Open Source Initiative says open weights alone fall short of open source AI. A card states a licence; it is not legal advice. Check the current documentation.

The tests check the hand-computed LoRA values, finite-difference gradients, every planted flaw, the split, the budget arithmetic and the dry run.

cd kit && python -m pytest tests/test_finetune_data.py tests/test_finetune_decide.py tests/test_finetune_lora.py tests/test_finetune_toy.py | tail -n 1 | sed 's/ in [0-9.]*s//'

31 passed

Where this shows up at work

Aiko in Osaka is asked whether a support model should be fine-tuned on last year's tickets (an illustrative case). Her team writes a decision record first: sampled baseline failures are two retrieval misses and one stale document, so the record says "fix the data first". The tickets also hold customer names, so "use last year's tickets" becomes "use a small set of rewritten examples". Later a measured prompt falls short on style and the record is reopened with evidence. Baseline, record, clean data, then experiment is also what a reviewer looks for in a fine-tuning portfolio piece.

Common mistakes

• Fine-tuning to fix facts. Facts change; weights cannot be cited, diffed or rolled back one fact at a time. Fix data and retrieval; keep numbers out of targets.
• Training without a baseline. Without saved scores and failures, "improved" is a feeling.
• Splitting rows instead of groups. Near-copies on both sides make validation measure memory. Split by group and check cross-split overlap.
• Tuning on the test set. Every choice made by looking at test makes it a second validation set. Choose on validation, score test once.
• Treating toy results as evidence. Rank, alpha and drift here come from a layer we designed; they show what to measure on a real model.

Try it yourself

1. Easy. Revisit Reliable Output Contracts and RAG Reliability and Evaluation Data. For three new stories, decide the route before you run anything: (a) answers still name the old courier after a change; (b) agents want one brand voice across forty reply types and a measured prompt falls short; (c) a quote adds cents wrongly. Then call route_for on stale_fact, style_inconsistent, arithmetic.
2. Medium. Revisit Evaluation Datasets and Deterministic Tests. Remove every row with an error from the flawed file, write the rest to a temporary file and validate it. Which rows survive, and what remains to read?
3. Hard. The readiness review. For scenarios s05 and s06, answer the eight questions as a reviewer who has measured a prompt and a first fix, but has no rollback gate for s05. Write a record choosing fine_tune for each and print its problems. What should s05's decision be, and why?

Answers

Easy. (a) data and retrieval, not weights; (b) a candidate, after the prompt baseline; (c) ordinary code.

from evidencedesk.finetune.decide import route_for

for symptom in ("stale_fact", "style_inconsistent", "arithmetic"):
    r = route_for(symptom)
    print(f"{symptom:<19}{r.fine_tune:<10}{r.first_fix}")

stale_fact         no        update the data, filter superseded versions, retrieve
style_inconsistent candidate a clearer prompt and examples, measured against a rubric
arithmetic         no        ordinary code behind a tool

Medium. Nine of eleven rows have an error; ex-001 and ex-f08 survive, and the second still carries a warning about a number in its target. A warning passes the gate and still needs a human read.

import json
import tempfile
from pathlib import Path

from evidencedesk.finetune.data import eval_texts, read_jsonl, validate

report = validate(FLAWED, eval_texts())
bad_lines = {p.line for p in report.errors}
kept = [obj for n, obj in read_jsonl(FLAWED) if n not in bad_lines]
with tempfile.TemporaryDirectory() as tmp:
    path = Path(tmp) / "kept.jsonl"
    path.write_text("".join(json.dumps(o) + "\n" for o in kept), encoding="utf-8")
    after = validate(path, eval_texts())
print("rows with an error:", len(bad_lines), "of", report.rows, "| kept:", [o["id"] for o in kept])
print(after.text())

rows with an error: 9 of 11 | kept: ['ex-001', 'ex-f08']
2 rows, 0 errors, 1 warnings
  line  2 ex-f08  warning number_in_target a number in a target can go stale

Hard. s05 is blocked: one open question (the rollback gate) leaves fine_tune refused; its decision stays "prompt" until a gate exists. s06 has no unmet question and the record passes, but passing the record is permission to run a small, measured experiment against the baseline, and a small classifier is still a rival to beat.

from evidencedesk.finetune.decide import QUESTIONS, DecisionRecord, readiness

cases = {  # scenario -> answers to the eight questions, as a reviewer might give them after measuring
    "s05": {k: True for k, _ in QUESTIONS} | {"rollback": False},
    "s06": {k: True for k, _ in QUESTIONS},
}
for sid, answers in cases.items():
    unmet = readiness(answers)
    rec = DecisionRecord(id=f"dec-{sid}", date="2026-10-13", problem=sid, symptom="x", options=["prompt", "fine_tune"],
                         decision="fine_tune", evidence=["baseline kestrel-v2 saved", "prompt and first fix measured"],
                         risks=["training data goes stale"], review_when="next release", baseline="kestrel-v2", unmet=unmet)
    print(sid, "unmet:", len(unmet), "| record problems:", rec.problems())

s05 unmet: 1 | record problems: ['fine_tune chosen with a baseline missing or readiness questions unmet']
s06 unmet: 0 | record problems: []

Pass checklist:

• Easy: no, candidate, no, each with a first fix other than weights.
• Medium: two rows kept, zero errors, one warning that you can explain.
• Hard: s05 shows one unmet question and a refusal; s06 shows none; you named the rollback gate as the missing piece.

Summary

• Fine-tuning changes behaviour, not changing facts. Name the cause first; most Kestrel failures were data, retrieval, contract or abstention problems.
• Choose the cheapest reversible fix, measured on the same evaluation set. Save a baseline (scores, latency, configuration, failures) and write a decision record; refuse fine_tune while readiness questions are open.
• Training data needs a schema, rights fields, group splits and a validator for duplicates, conflicting targets, leakage, secrets and personal data. A clean report is necessary, not sufficient.
• LoRA trains two thin matrices beside a frozen weight: W + (alpha / r) B A. Rank trades capacity for parameters; merged and unmerged outputs are equal, which a hand check proves.
• Toy training shows overfitting (validation turns up while training falls) and drift, and that an adapter does not promise to forget less. Estimates and dry-run reports stay labelled hypothetical, and a missing number stays missing.

Next: PEFT Experiments and Model Rights uses this code to inspect a real adapter configuration, count trainable parameters where the environment allows, evaluate before and after on the same holdout, check overfitting and licences, and decide whether to keep or roll back an adapter.