Glossary
Fine-tuning
Further training of a pretrained model on a smaller, focused dataset to adapt it to a task, domain, or style. Supervised fine-tuning (SFT) is the most common form, using curated example inputs and outputs.
Fine-tuning corpora are small next to pretraining — thousands to millions of examples rather than trillions of tokens — but the quality bar is far higher. Common forms include supervised fine-tuning on instruction–response pairs, domain adaptation on specialist material, and parameter-efficient methods such as LoRA. Errors in a small corpus propagate, so curation and review matter most here.
Why it matters
Fine-tuning is where niche, expert, well-documented datasets earn a premium over bulk data.