Skip to content

ADAPT-Q

Specializing language models without making them forget.

The problem

Fine-tuning a general-purpose language model on a specialized domain such as medicine, law, finance or science makes it better at that domain and measurably worse at much else. The more different the domain, the larger the loss. For teams deploying models in regulated or high-stakes settings, this is more than a benchmark curiosity: behavior that was tested and approved before tuning can quietly degrade after it.

What we're building

ADAPT-Q is a family of fine-tuning methods aimed at that trade-off: learning a demanding new domain as well as standard approaches do, while preserving substantially more of the model's general capability. We are evaluating it across several base models and four distinct professional domains, against established fine-tuning methods and forgetting-mitigation baselines.

How we're doing it

We treat this as a measurement problem first:

  • Pre-registered study design. Hypotheses, metrics, statistical tests and exclusion rules are fixed and registered before any confirmatory experiment runs.
  • A purpose-built, fully tested research harness. Reproducible runs with pinned models and datasets, full provenance for every result, and automated checks that catch experiments that silently did nothing.
  • Fair comparisons. Methods are compared at matched learning and matched resource budgets, with controls that separate a real effect from an artefact of setup.
  • Openly licensed, document-level data. Decontaminated splits, so evaluation reflects generalization rather than memorization.

A companion line of research explores how structured domain knowledge, such as knowledge graphs, can guide what a model learns during specialization.