Fine-tuning
SFT · supervised fine-tuning · LoRA
ひとことで
Fine-tuning continues training a pre-trained model on a smaller, task-specific dataset, adjusting its weights so it reliably produces a particular format, tone or narrow skill. It is a poor way to teach facts, which change and are better supplied by retrieval at request time.
Fine-tuning starts from a model that already speaks the language and nudges its weights toward a specific behaviour. The right question to ask before starting is: is my problem knowledge, or behaviour?
If the model does not know your product catalogue, fine-tuning is the wrong tool — the catalogue changes weekly and retraining does not. Use retrieval.
If the model knows plenty but will not consistently emit your JSON schema, adopt your house style, or apply your internal classification taxonomy, fine-tuning does that well and cheaply.
Full fine-tuning updates every parameter: expensive, and it produces a complete model copy to host. LoRA and similar parameter-efficient methods train a small adapter alongside the frozen base model, which is dramatically cheaper and sufficient for most purposes.
Three practical cautions. Fine-tuned models can lose general capability in areas the training set neglects — a phenomenon usually called catastrophic forgetting. The base model underneath will eventually be deprecated, and your adapter goes with it, so budget for redoing the work. And the effort is frequently unnecessary: teams routinely fine-tune to solve problems that a clearer prompt and a few examples in context would have solved in an afternoon. Try that first.
よくある質問
- How many examples do I need to fine-tune?
- For format and tone, often a few hundred high-quality examples outperform tens of thousands of noisy ones. Consistency in the training set matters more than volume.
- What is LoRA?
- Low-Rank Adaptation trains a small set of additional parameters instead of updating the whole model. It is far cheaper, produces a small adapter file rather than a full model copy, and is adequate for most customisation work.