Fine-tuning on a laptop: a field tutorial

You don’t need a cluster to build intuition.¹ A 3B model, LoRA adapters, and an evening are enough. Start by pinning your dependencies:

model = load("gemma-3b")
cfg = LoraConfig(r=16, alpha=32)

The rank parameter is where most people over-spend. Sixteen is plenty for style transfer; go higher only when the task genuinely changes what the model must represent.

Getting data into shape

Most of the time you spend on a fine-tune should go into the dataset, not the config. For a style-transfer job — teaching the model to write in a particular voice, say — a few hundred clean examples beats a few thousand noisy ones. Keep the format boring and consistent: a prompt field, a completion field, nothing clever.

import json

def load_examples(path):
    rows = [json.loads(line) for line in open(path)]
    return [{"prompt": r["prompt"], "completion": r["completion"]} for r in rows]

With the config and data in hand, the training loop itself is short. This is deliberate — if your loop is long, the complexity almost always belongs in the data pipeline, not here.

from train import model, cfg
from dataset import load_examples

data = load_examples("examples.jsonl")
model.attach_lora(cfg)
model.fit(data, epochs=3, lr=2e-4)
model.save_adapter("out/adapter")

Three epochs is a reasonable default for a few hundred examples on a small model — enough for the style to take, not so much that the model starts memorizing individual completions. Watch the loss curve rather than trusting the epoch count blindly; if it’s still dropping sharply at epoch three, let it run one more.

Once training finishes, merge the adapter back into the base weights before you ship anything, so downstream code doesn’t need to know a LoRA was ever involved:

from train import model

model.load_adapter("out/adapter")
merged = model.merge_and_unload()
merged.save_pretrained("out/merged-model")

That’s the whole loop → pin the base model, keep the dataset small and clean, train for a handful of epochs, merge, evaluate on a few held-out prompts by hand before you trust it on anything real. None of this requires a rented GPU. It requires patience with the data.