Fine-tuning is the process of taking a pre-trained foundation model and continuing its training on a curated, task-specific dataset to permanently modify its weights — encoding new behaviors, domain knowledge, output formats, or stylistic patterns that persist across all future interactions. Unlike prompt engineering, which adjusts inputs at inference time, fine-tuning changes the model itself. The result is a specialized model that produces better outputs on the target task while retaining most of the general capabilities it learned during pre-training.
Fine-tuning sits at the center of a critical decision in every AI application: should you use a powerful general-purpose model with careful prompting, or a smaller specialized model tuned for your specific use case? The answer has significant implications for quality, cost, latency, and data privacy — and it has shifted over time as foundation models have grown more capable and prompting techniques have matured.
How fine-tuning works
All fine-tuning follows the same fundamental pattern: expose the model to examples of desired behavior and adjust its weights to make that behavior more likely.
The training process
-
Dataset preparation. Curate a dataset of input-output examples that demonstrate the desired behavior. For supervised fine-tuning (SFT), this typically means instruction-response pairs in a conversational format. Quality matters more than quantity — 500 high-quality examples often outperform 10,000 noisy ones.
-
Training configuration. Set hyperparameters: learning rate (typically 1e-5 to 5e-5, much lower than pre-training), batch size, number of epochs (typically 1-5), and which layers or parameters to update. Too-high learning rates or too many epochs cause catastrophic forgetting — the model loses its general capabilities.
-
Training execution. The model processes each example, computes the loss (how far its output was from the desired output), and adjusts its weights via backpropagation. For full fine-tuning of a 70B parameter model, this requires multiple high-end GPUs and hours to days of compute. For parameter-efficient methods like LoRA, a single GPU suffices.
-
Evaluation. Test the fine-tuned model on a held-out validation set and compare it against the base model with the best available prompt. If the fine-tuned model doesn’t meaningfully outperform the prompted base model, the fine-tuning dataset or configuration needs adjustment.
Full fine-tuning vs. parameter-efficient fine-tuning
The distinction between these approaches has become the defining architectural decision in production fine-tuning:
| Approach | What it updates | GPU memory needed (70B model) | Training time | Quality ceiling |
|---|---|---|---|---|
| Full fine-tuning | All parameters | 8-16x A100/H100 80GB GPUs | 12-48 hours | Highest — can modify any behavior |
| LoRA | Low-rank adapters (0.1-2% of params) | 1-2x A100/H100 80GB GPUs | 2-8 hours | Near-full for most tasks |
| QLoRA | LoRA on 4-bit quantized model | 1x GPU with 24-48GB VRAM | 3-12 hours | Slightly below LoRA; accessible on consumer hardware |
| Prefix tuning | Learned prompt embeddings | 1x GPU with 16-24GB VRAM | 1-4 hours | Good for style/format; limited for knowledge |
| Adapters | Small inserted networks | 1x GPU with 24-48GB VRAM | 2-6 hours | Good general-purpose; less popular than LoRA |
LoRA (Low-Rank Adaptation) has become the dominant method. Introduced by Hu et al. in 2021, it freezes all original model weights and inserts small trainable matrices into each Transformer layer. These matrices are factored into low-rank components — typically rank 8-64 — that capture task-specific patterns with minimal parameter overhead. A LoRA adapter for a 70B model might contain only 100-500 million trainable parameters (0.1-0.7% of total), yet achieve 95-99% of full fine-tuning quality on most tasks.
QLoRA (Dettmers et al., 2023) combines LoRA with 4-bit quantization of the base model, reducing memory requirements enough to fine-tune a 70B model on a single 48GB GPU. This democratized fine-tuning, making it accessible to individual researchers and small teams.
The fine-tuning decision framework
Deciding when to fine-tune versus when to use prompt engineering is the most important architectural decision in many AI applications. A systematic framework:
When prompt engineering is sufficient
- The task is well-defined and the model understands it with clear instructions
- Output quality with optimized prompts meets requirements
- The input-output mapping can be communicated through 3-8 examples (few-shot)
- You need to iterate quickly (prompt changes are instant; fine-tuning takes hours)
- Your data is too sensitive to upload to a training API
- You are using a frontier model that cannot be fine-tuned (e.g., Claude Opus 4 as of mid-2026)
When fine-tuning is necessary
- Consistent output format. The model needs to reliably produce a specific schema, tone, or style across thousands of outputs. Fine-tuning encodes format compliance into the weights rather than relying on instruction following.
- Domain specialization. The model needs to understand domain-specific terminology, reasoning patterns, or conventions that are underrepresented in its pre-training data (specialized legal codes, proprietary product catalogs, niche technical domains).
- Latency reduction. A fine-tuned smaller model (7-13B parameters) can match a frontier model’s quality on a specific task while running 5-20x faster and at a fraction of the cost. In latency-sensitive applications (autocomplete, real-time chat), this matters enormously.
- Cost reduction. For high-volume applications, replacing expensive frontier model API calls with a fine-tuned smaller model can reduce costs by 90%+. This is the primary economic motivation for fine-tuning in production.
- Behavioral modification. Teaching the model to consistently refuse certain types of requests, adopt a specific persona, or follow company-specific guidelines that resist prompt engineering approaches.
The decision matrix
| Scenario | Recommended approach | Reasoning |
|---|---|---|
| Building a prototype | Prompt engineering | Iterate fast, no training needed |
| 100 queries/day, varied tasks | Prompt engineering with frontier model | Volume too low to justify fine-tuning cost |
| 100,000 queries/day, specific task | Fine-tune smaller model | Cost savings justify the investment |
| Specialized domain (legal, medical) | RAG + prompt engineering first; fine-tune if RAG is insufficient | RAG handles most domain knowledge needs |
| Strict output format compliance | Fine-tune | Format compliance from fine-tuning exceeds prompt-based compliance |
| Matching a specific writing style | Fine-tune | Style transfer requires weight-level learning |
Fine-tuning across providers
Every major model provider and open-source ecosystem offers fine-tuning capabilities, but with significant differences in flexibility, cost, and model access:
Commercial fine-tuning APIs
| Provider | Models available | Min examples | Training cost | Key limitation |
|---|---|---|---|---|
| OpenAI | GPT-4o, GPT-4o mini, GPT-4.1 mini | 10 (recommended: 50-100) | ~$25 per 1M training tokens (GPT-4o mini) | No weight access; model served by OpenAI only |
| Google (Vertex AI) | Gemini 1.5 Flash, Gemini 2.0 Flash | 10-500 | Varies by model and region | Limited model selection for fine-tuning |
| Cohere | Command R+ | 2 | Custom pricing | Enterprise-focused |
| Mistral (La Plateforme) | Mistral Small, Mistral Large | 1 | ~$4 per 1M training tokens (small) | Fewer model options than OpenAI |
Open-source fine-tuning
Open-weights models offer the most flexibility because you can fine-tune them yourself on your own infrastructure:
| Model family | Popular fine-tuning targets | Community tooling |
|---|---|---|
| Llama 4 (Meta) | 8B Scout, 109B Maverick | Unsloth, Axolotl, TRL, LLaMA-Factory |
| Mistral | 7B, Mixtral 8x7B | Mistral-finetune, Axolotl |
| Qwen 2.5 (Alibaba) | 7B, 32B, 72B | LLaMA-Factory, TRL |
| Gemma 2 (Google) | 9B, 27B | Keras, TRL |
| Phi-4 (Microsoft) | 14B | TRL, Unsloth |
Unsloth has emerged as the most popular fine-tuning framework for practitioners, offering 2x training speedups and 60% memory reduction through optimized kernels, with a simple interface that handles LoRA/QLoRA configuration automatically. Axolotl provides more configuration flexibility for advanced users. TRL (Transformer Reinforcement Learning) from Hugging Face is the standard library for both SFT and RLHF fine-tuning.
Fine-tuning dataset design
The dataset is the most important variable in fine-tuning quality. Models learn from patterns in the training data with unforgiving precision — they will replicate every consistent pattern, including unintended ones.
Dataset size guidelines
| Use case | Recommended examples | Rationale |
|---|---|---|
| Output format training | 50-200 | Format patterns are simple and learned quickly |
| Style/tone adaptation | 200-500 | Style requires more diverse examples to generalize |
| Domain knowledge | 500-2,000 | Knowledge requires broader coverage |
| Complex task specialization | 1,000-10,000 | Complex behavior needs many examples with edge cases |
| Instruction-following base model | 50,000-500,000 | General instruction-following requires large, diverse datasets |
Quality over quantity
A 2024 study from Allen AI (“LIMA: Less Is More for Alignment”) demonstrated that fine-tuning Llama on just 1,000 carefully curated examples produced outputs that human evaluators preferred over models fine-tuned on 52,000 lower-quality examples. The insight: each example should represent exactly the behavior you want the model to learn. A single low-quality example that violates your desired format can degrade format compliance across thousands of outputs.
Synthetic data for fine-tuning
Using a frontier model (Claude Opus 4, GPT-4.1) to generate training data for fine-tuning a smaller model is now the dominant approach for creating fine-tuning datasets. This process — called distillation — works as follows:
- Design prompts that make the frontier model produce ideal outputs for your task
- Generate 500-5,000 examples by running diverse inputs through the frontier model
- Manually review and correct the top 10-20% of examples for quality
- Fine-tune the target model (typically 7-13B parameters) on this dataset
- Evaluate against the frontier model on a held-out test set
This approach regularly produces fine-tuned small models that achieve 85-95% of the frontier model’s quality on the target task, at 10-50x lower inference cost and 5-20x lower latency.
Common dataset mistakes
- Homogeneous examples. If all examples follow the same pattern, the model overfits to that pattern and fails on variation. Include diverse inputs, edge cases, and boundary conditions.
- Inconsistent formatting. If example outputs use different formats (sometimes bullet points, sometimes paragraphs), the model will randomly switch between them. Pick a format and stick to it.
- Teaching facts through fine-tuning. Fine-tuning is poor at injecting factual knowledge — RAG is far more effective. Use fine-tuning for behavior and style, not for teaching the model new facts.
- Insufficient context. If training examples lack the context the model will have at inference time (system prompts, retrieved documents), the model may learn patterns that break in production.
Fine-tuning vs. alternatives
Fine-tuning competes with several alternative approaches for model specialization:
| Approach | Changes weights? | Training data needed | Time to deploy | Best for |
|---|---|---|---|---|
| Prompt engineering | No | 0 (or a few examples in-context) | Minutes | Rapid iteration, prototyping, general tasks |
| Fine-tuning (LoRA) | Yes (partially) | 50-10,000 examples | Hours | Format, style, domain specialization |
| Fine-tuning (full) | Yes (all) | 500-100,000 examples | Days | Maximum quality, complex behavior modification |
| RAG | No | Knowledge documents | Hours | Factual accuracy, dynamic knowledge |
| Distillation | Yes (student model) | Teacher model outputs | Hours-days | Cost reduction, deploying smaller models |
| Continued pre-training | Yes (all) | Millions of tokens of domain text | Days-weeks | Deep domain adaptation (e.g., medical, legal corpora) |
The most effective production systems often combine multiple approaches: continued pre-training on domain text, followed by SFT on task-specific examples, followed by prompt engineering for deployment-specific behavior, with RAG for factual grounding.
The economics of fine-tuning
Fine-tuning involves upfront training costs that amortize over inference volume:
Training costs:
- Fine-tuning GPT-4o mini on 1,000 examples (~500K tokens): ~$12
- Fine-tuning Llama 4 8B with QLoRA on cloud GPU (4 hours on A100): ~$8-16
- Fine-tuning Llama 4 8B with QLoRA on consumer GPU (RTX 4090, 6 hours): ~$3 electricity
Break-even analysis: If fine-tuning a 7B model lets you replace a frontier API at $10/1M output tokens with self-hosted inference at $0.50/1M output tokens, and you process 10M output tokens/month, you save $95/month. The fine-tuning cost ($10-50) pays for itself within the first week.
Ongoing costs: Fine-tuned models on commercial APIs (OpenAI) typically cost 2-6x the base model’s per-token price. Self-hosted fine-tuned open-source models cost only inference compute — no per-token premium. This makes self-hosted open-source models the clear economic winner for high-volume fine-tuning use cases.
RLHF and preference-based fine-tuning
Beyond supervised fine-tuning, more advanced fine-tuning methods align models with human preferences:
RLHF (Reinforcement Learning from Human Feedback). Human raters rank model outputs by quality. A reward model is trained on these preferences, then used to fine-tune the base model via reinforcement learning (PPO). This is how ChatGPT, Claude, and Gemini are trained to be helpful and harmless — it is not typically available to end users.
DPO (Direct Preference Optimization). An alternative to RLHF that skips the reward model step. Instead of training a separate reward model, DPO directly optimizes the language model on preference pairs (chosen response vs. rejected response). Simpler, cheaper, and increasingly popular for both commercial alignment and custom fine-tuning. Available through Hugging Face TRL.
ORPO (Odds Ratio Preference Optimization). A newer approach that combines SFT and preference optimization in a single training step, further simplifying the process. Shown to match DPO quality with lower computational cost in several studies.
Frequently asked questions
When should I fine-tune vs. use prompt engineering? Start with prompt engineering — it is faster, cheaper, and reversible. Fine-tune only when: (1) prompt engineering cannot achieve the required quality after systematic optimization, (2) you need consistent output format or style across high-volume production use, (3) you want to replace an expensive frontier model with a cheaper fine-tuned smaller model, or (4) latency requirements demand a smaller, faster model. Most applications never need fine-tuning; careful prompting with a frontier model is sufficient.
How much data do I need for fine-tuning? It depends on the task. Format and style training requires 50-200 examples. Domain specialization requires 500-2,000. Complex task specialization may need 1,000-10,000. Quality matters far more than quantity — 200 carefully curated examples typically outperform 5,000 noisy ones. Start with the minimum viable dataset and add examples only when evaluation reveals specific failure modes.
Can I fine-tune Claude or GPT-4? As of mid-2026, Anthropic does not offer public fine-tuning for Claude models (though enterprise agreements may include it). OpenAI offers fine-tuning for GPT-4o, GPT-4o mini, and GPT-4.1 mini through their API. Google offers fine-tuning for select Gemini models through Vertex AI. For maximum fine-tuning flexibility, open-weights models like Llama 4 and Mistral can be fine-tuned on your own infrastructure with no restrictions.
What is catastrophic forgetting? Catastrophic forgetting occurs when fine-tuning causes the model to lose general capabilities it learned during pre-training. The model becomes good at the fine-tuning task but worse at everything else. It is caused by too-high learning rates, too many training epochs, or insufficiently diverse training data. LoRA mitigates catastrophic forgetting by design — since it only modifies a small fraction of parameters, the base model’s general capabilities are largely preserved.
What is the difference between fine-tuning and training from scratch? Training from scratch (pre-training) creates a model from randomly initialized weights using billions or trillions of tokens of text — costing millions of dollars and weeks to months of compute. Fine-tuning starts from an already-capable pre-trained model and adapts it using a much smaller dataset (hundreds to thousands of examples) at a fraction of the cost (often under $100). Fine-tuning is essentially transfer learning: leveraging the broad knowledge from pre-training and specializing it for a specific task.