Fine-Tuning LLMs: A Practical Guide

1What Is Fine-Tuning?

Fine-tuning is the process of continuing to train a pre-trained model on a smaller, domain-specific dataset to adapt its behaviour for a particular task. The model starts with broad language understanding from pre-training — billions of tokens of general text — and then specialises on your data.

Think of it like this: pre-training is a university education; fine-tuning is a professional apprenticeship in a specific role. The foundations are already there, and fine-tuning builds the specialist layer on top.

2When to Fine-Tune vs Prompt vs RAG

Choose your approach based on what you need to change in the model's behaviour. Each option trades off cost and time to deploy against how deeply it reshapes the model.

Fine-tune when prompting consistently fails to produce the output style you need — not before. Common legitimate use cases include a legal drafting assistant trained on your firm's contract templates, a code assistant fine-tuned on your company's coding conventions, and a customer support model trained on your support ticket history.

• Prompting — best for simple tasks and one-off queries · cost: zero · time to deploy: immediate
• RAG — best for factual Q&A over your documents · cost: low · time to deploy: hours
• Fine-tuning — best for consistent style, format, and domain vocab · cost: medium · time to deploy: days
• Full pre-training — best for a new language or entirely new domain · cost: very high · time to deploy: months

3Types of Fine-Tuning

There are several distinct approaches to fine-tuning, ranging from updating every weight to training a tiny adapter or aligning the model with human preferences. The right one depends on your budget, hardware, and goal.

• Full fine-tuning: update all model weights on your dataset. Most effective but requires significant GPU memory (a 70B model needs ~140GB VRAM). Rarely practical without a large cluster.
• Parameter-Efficient Fine-Tuning (PEFT): freeze most weights and train a small subset or add small adapter layers. LoRA is the most popular PEFT method.
• Instruction fine-tuning: train on (instruction, response) pairs to improve the model's ability to follow diverse instructions. Produces chat/instruction models from base models.
• RLHF (Reinforcement Learning from Human Feedback): use human preference rankings to align model outputs with human preferences. Used to train models like Claude and ChatGPT from instruction-tuned base models.

4LoRA: Efficient Adaptation

LoRA (Low-Rank Adaptation) is the standard efficient fine-tuning method. Instead of updating all model weights, it injects small trainable matrices into specific weight layers of the transformer.

A 7B model that needs ~14GB VRAM for full fine-tuning can be LoRA fine-tuned with ~6GB — making it feasible on a single consumer GPU or a small cloud instance.

• The original weight matrix W is frozen and not trained.
• Two small matrices A and B are added: the adapter computes W + AB.
• Only A and B are trained, and they have far fewer parameters than W.
• At inference time, AB can be merged into W with zero latency overhead.

5Preparing Your Dataset

Dataset quality is the single most important factor in fine-tuning success. The model learns from every example, including bad ones, so curation matters more than raw volume.

For instruction fine-tuning, format each example as an instruction, an input, and the expected output. Quantity needs vary by task complexity, but the targets below are a useful starting point.

• Simple format/style — minimum examples: 100–500 · good target: 1,000–5,000
• Domain adaptation — minimum examples: 500–2,000 · good target: 5,000–20,000
• New capability — minimum examples: 2,000–10,000 · good target: 50,000+

{
  "instruction": "Summarise the following support ticket in one sentence.",
  "input": "Customer says their order #12345 arrived damaged...",
  "output": "Customer received damaged order #12345 and requests replacement."
}

6Dataset Quality Checklist

Before you start training, run your dataset through a quality checklist. Most fine-tuning failures trace back to the data, not the hyperparameters.

• Diverse inputs: cover the full range of inputs the model will see in production. A dataset of 10,000 nearly identical examples teaches less than 1,000 varied ones.
• Consistent output style: if the target output style varies across examples, the model learns inconsistency. Standardise formatting before training.
• No hallucinated outputs: every output in the training set should be verifiably correct. Bad examples teach bad behaviour.
• Train/eval split: hold out 10–20% for evaluation, and never evaluate on training data.
• Remove PII: personally identifiable information in training data becomes part of the model. Scrub it before training.

7Fine-Tuning with Hugging Face PEFT

The Hugging Face stack — transformers, peft, datasets, and trl — makes a LoRA fine-tuning job a short script. Load the base model in 4-bit quantisation to save GPU memory, then wrap it with a LoRA configuration.

Targeting only the query and value projections keeps the trainable parameter count tiny — roughly 4M of 3.2B parameters, about 0.1% — which is what makes the memory savings so dramatic.

# Install dependencies
pip install transformers peft datasets accelerate bitsandbytes

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
from datasets import load_dataset

# Load base model in 4-bit quantisation (saves GPU memory)
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    load_in_4bit=True, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Configure LoRA adapter
lora_cfg = LoraConfig(
    r=16,                # rank: higher = more expressive, more memory
    lora_alpha=32,       # scaling factor
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    task_type=TaskType.CAUSAL_LM
)
model = get_peft_model(model, lora_cfg)
model.print_trainable_parameters()
# Trainable: ~4M of 3.2B params (0.1%) — huge memory saving

8Choosing a Base Model

Your choice of base model shapes capability, hardware needs, and licensing. The open-weight models below cover the most common fine-tuning scenarios.

For most fine-tuning tasks, a 7B model fine-tuned well outperforms a 70B model prompted poorly. Start small, iterate on data quality, then scale up model size only if needed.

• Llama 3.2 3B/8B — 3B / 8B params · best for general tasks on a consumer GPU · licence: Open (commercial)
• Mistral 7B — 7B params · best for efficient, strong reasoning · licence: Open (Apache 2.0)
• Phi-3.5 Mini — 3.8B params · best for on-device, resource-constrained use · licence: Open (MIT)
• Qwen 2.5 7B/14B — 7B / 14B params · best for multilingual and code · licence: Open (commercial)
• Gemma 2 9B — 9B params · best for the Google ecosystem, safety tuned · licence: Open (commercial)

9Training Parameters That Matter

A handful of training arguments drive most of your results. Learning rate, number of epochs, and LoRA rank are the levers you will tune most often.

Key parameters to watch: learning rate (too high causes forgetting, too low means no learning), epochs (too many overfits small datasets), and LoRA rank r (higher is more expressive but uses more memory).

training_args = TrainingArguments(
    output_dir="./ft-output",
    num_train_epochs=3,              # usually 2-5 for instruction tuning
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,   # effective batch = 16
    learning_rate=2e-4,              # typical LoRA range: 1e-4 to 3e-4
    warmup_ratio=0.03,
    lr_scheduler_type="cosine",
    logging_steps=10,
    save_strategy="epoch",
    evaluation_strategy="epoch",
    fp16=True,                       # mixed precision for speed
)

10Evaluating Your Fine-Tuned Model

Evaluation should combine automatic metrics with human judgment and regression checks. No single number tells you whether a fine-tuned model is ready to ship.

• Task-specific metrics: BLEU/ROUGE for summarisation, F1 for classification, pass@k for code generation.
• Human evaluation: have domain experts rate outputs blind, not knowing which is fine-tuned versus base. More reliable than automatic metrics for open-ended generation.
• Side-by-side comparison: run identical prompts through the base model and fine-tuned model, then compare outputs directly.
• Regression testing: ensure the model still handles general tasks correctly, not just your specific domain.

11Deploying a Fine-Tuned Model

Once trained, a LoRA adapter is small — typically 100MB to 1GB — so you can save it separately and merge it into the base weights at load time for inference.

Deployment options include Hugging Face Inference Endpoints (managed, pay-per-request), Together AI or Replicate (hosted fine-tuned models), or self-hosting on a GPU instance such as RunPod, Lambda Labs, or your own hardware.

# Save the LoRA adapter (small — typically 100MB-1GB)
model.save_pretrained("./lora-adapter")

# Load and merge for inference
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
model = PeftModel.from_pretrained(base, "./lora-adapter")
model = model.merge_and_unload()   # merge adapter into base weights

12Key Takeaways

Fine-tuning is most valuable for shaping how a model behaves, and it works best alongside RAG for factual grounding.

• Fine-tune for style, format, and domain vocabulary; use RAG for factual grounding. They complement each other.
• LoRA makes fine-tuning affordable: train less than 1% of parameters and get 80%+ of full fine-tuning performance.
• Dataset quality is the most important variable — 1,000 excellent examples beat 10,000 mediocre ones.
• Always hold out an evaluation set, monitor for catastrophic forgetting, and compare to the base model before deploying.

13What to Learn Next

To go deeper in LLM customisation, build on fine-tuning with the techniques that complement it.

• RAG Explained — combine with fine-tuning for style plus factual accuracy.
• AI Agents Explained — deploy your fine-tuned model as an agent.
• Vector Databases Explained — build the RAG complement to your fine-tuned model.

14Frequently Asked Questions

How much GPU do I need to fine-tune a 7B model? With 4-bit quantisation and LoRA, a 7B model can be fine-tuned on a single 24GB GPU (RTX 3090/4090 or equivalent). Cloud options include a RunPod A100 (40GB) at ~$1.5/hour, or Google Colab Pro with an A100 for short runs. Expect 1–4 hours for 1,000–10,000 examples on a 7B model.

Can I fine-tune Claude or GPT-4? GPT-4o and GPT-3.5-Turbo can be fine-tuned via the OpenAI API, and Claude fine-tuning is available through the Anthropic API for enterprise customers. Both are significantly more expensive than fine-tuning an open-source model yourself, but require no GPU infrastructure.

How many training examples do I need? For simple style or format tasks, 100–1,000 high-quality examples can produce noticeable improvement; for complex domain adaptation, expect 5,000–50,000. Quality consistently outweighs quantity — a dataset of 500 carefully curated examples often outperforms 5,000 scraped, inconsistent ones.

What is QLoRA? QLoRA (Quantised LoRA) combines 4-bit quantisation of the base model with LoRA adapters. It achieves nearly the same results as LoRA but uses significantly less GPU memory, enabling fine-tuning of larger models on smaller hardware. It's what the load_in_4bit=True option in the example above implements.