Research · Model Adaptation
Adapt behavior with the smallest justified intervention.
LoRA, QLoRA, distillation, and full fine-tuning solve different problems and create different risks. We start with the measured behavior gap, preserve lineage, and increase intervention only when evidence justifies it.
Parameter-efficient does not mean evaluation-free.
Intervention Ladder
Escalate only as far as the problem requires.
Policy & prompt
Clarify instructions, output contracts, and refusal boundaries when the model already has the needed capability.
Context & retrieval
Supply fresh, attributable knowledge without embedding changing facts into model weights.
Tools & workflow
Move deterministic actions, validation, and external state into explicit software controls.
LoRA or QLoRA
Adapt a defined behavior with a compact, separable artifact tied to an exact base and serving profile.
Distillation
Train a student from reviewed teacher-generated examples when footprint, latency, or specialization justifies the added data pipeline.
Full fine-tuning
Change the broader parameter set only when narrower interventions cannot meet the measured objective.
Methods
What each adaptation method means.
LoRA
Low-Rank Adaptation trains small additional matrices while leaving the base model largely unchanged. The resulting adapter is compact, reversible, and inseparable from the identity of the base model, tokenizer, template, and target modules used to create it.
QLoRA
QLoRA applies LoRA-style training while the base is held in a quantized representation to reduce the training footprint. That efficiency does not remove the need to qualify the adapter, merge or conversion process, and final runtime.
Distillation
Distillation uses a stronger teacher to help train a smaller or more specialized student. Teacher output is untrusted labeled data: it needs provenance, filtering, human review, leakage controls, and independent evaluation.
Retrieval versus adaptation
Retrieval is usually the better tool for changing facts, private knowledge, and attributable evidence. Weight adaptation is better reserved for persistent behavior, style, format, or task patterns that survive held-out evaluation.
Evidence Chain
An adapter is a release artifact, not a loose file.
- Exact base identity, license, tokenizer, and template
- Immutable dataset snapshot and exclusion rules
- Training configuration and bounded canary record
- Adapter lineage and compatibility metadata
- Held-out candidate-versus-baseline evaluation
- Merge, conversion, quantization, and serving checks
- Model card, limitations, decision, and rollback target
Current Status
Methodology published; outcomes remain gated.
The reviewed materials support LoRA and parameter-efficient adaptation planning, reproducible data controls, validation, and a bounded canary workflow.
Distillation is presented here as a research method and optional future direction. We are not claiming a completed distilled model or a release-qualified adapted model.
Follow the Evidence