Training

Fine-tuning

ActivePublished

Key innovation

Instead of training from scratch, fine-tuning reuses knowledge from pretraining and adapts a ready model to a specific task or domain at a fraction of the data, time and compute.

How it works

1) Choose a pretrained base model. 2) Prepare a smaller task- or domain-specific dataset (e.g. instruction-response pairs for instruction tuning). 3) Continue training with gradient descent, usually at a low learning rate (e.g. around 2e-5) to avoid destroying knowledge from pretraining. 4) In full fine-tuning all weights are updated; in PEFT/LoRA the base weights are frozen and only injected low-rank adaptation matrices are trained. 5) The resulting model (or a lightweight adapter) is saved and deployed. A low learning rate, few epochs and regularization mitigate catastrophic forgetting and overfitting.

Problem solved

Training large models from scratch is expensive, requires huge datasets and compute, and a general-purpose model is rarely optimal for a narrow task or domain. Fine-tuning lets you reuse the knowledge captured in a base model and adapt it to a specific goal at low cost.

Components

Pretrained base modelStarting point - provides learned weights and general knowledge.

The pretrained model whose weights are further tuned (or frozen in PEFT).

Official

Task-specific datasetLearning signal that adapts the model to the task/domain.

A smaller, specialized dataset (e.g. input-output pairs) defining the target behavior.

Official

Loss function and optimizerDrives weight updates via gradient descent.

Typically cross-entropy loss and an Adam/AdamW-type optimizer with a low learning rate.

Official

Adapter modules (optional, PEFT)In PEFT/LoRA these are the only trained parameters; base weights stay frozen.

Small, low-rank matrices injected into model layers (LoRA) instead of updating all weights.

Official

Implementation

Reference implementations

Hugging Face Transformers (Trainer)

Python · Hugging Face

Official

Hugging Face PEFT (LoRA, QLoRA)

Python · Hugging Face

Official

Implementation pitfalls

Catastrophic forgettingHigh

Overly aggressive fine-tuning overwrites general knowledge acquired during pretraining.

Fix:Low learning rate, fewer epochs, PEFT/LoRA, mixing general with domain data.

Overfitting on a small datasetHigh

A small fine-tuning dataset encourages memorizing examples instead of generalizing.

Fix:Early stopping, regularization, validation, limiting the number of epochs.

Learning rate too highMedium

A high learning rate destabilizes learned weights and destroys base knowledge.

Fix:Use small values (e.g. ~2e-5) and learning-rate warm-up.

Evolution

2014

Fine-tuning in computer vision

Fine-tuning ImageNet-pretrained CNNs becomes standard transfer-learning practice in computer vision.

2018

ULMFiT

Inflection point

Howard and Ruder introduce universal language model fine-tuning for NLP, showing effective inductive transfer for any text task.

Pretraining (concept)Universal Language Model Fine-tuning for Text Classification (paper)

2018

BERT - pretrain-then-finetune paradigm

Inflection point

BERT popularizes the scheme where a pretrained model is fine-tuned with one extra layer for many NLP tasks, reaching SOTA results.

Transformer (concept)BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (paper)

2021

LoRA - parameter-efficient fine-tuning

Inflection point

LoRA freezes base weights and trains low-rank matrices, cutting trainable parameters by up to ~10,000x and GPU memory by 3x vs full fine-tuning of GPT-3 175B.

PEFT / LoRA (concept)LoRA: Low-Rank Adaptation of Large Language Models (paper)

2022

InstructGPT - instruction tuning

Inflection point

OpenAI fine-tunes GPT-3 on demonstrations and feedback (SFT + RLHF) so the model follows user instructions - the basis of modern assistants.

Instruction Tuning (concept)Training language models to follow instructions with human feedback (paper)

Hyperparameters (configurable axes)

Learning rateCritical

Too high destroys pretrained knowledge; typically around 1e-5-5e-5 for full fine-tuning.

2e-5Typical value in Hugging Face Trainer docs.

Trainable parameter scopeCritical

Full fine-tuning (all weights) vs PEFT/LoRA (a fraction of parameters). Determines memory cost and forgetting risk.

fullUpdates all parameters.

LoRALoRA cuts trainable parameters by up to ~10,000x for GPT-3 175B.

Number of epochsHigh

Too many epochs lead to overfitting on a small fine-tuning dataset.

1-3Typical range for LLM fine-tuning.

LoRA rank (PEFT variant)Medium

Size of the low-rank adaptation matrices; a trade-off between capacity and parameter count.

8Common default rank r for LoRA.

Compute bottleneck

GPU memory (optimizer states and activations)

Full fine-tuning must store gradients and optimizer states for all parameters, which is the main reason parameter-efficient methods (PEFT/LoRA) emerged.

Execution paradigm

Primary mode

Dense

Standard fine-tuning updates dense weights via backpropagation; in PEFT/LoRA only a small dense set (adapters) is updated while the rest stays frozen.