Case study 6 of 6
Fine-tuning legal LLMs
QLoRA and LoRA adapters for Qwen2.5-3B, trained on legal question answering and scored against LegalBench, the standard benchmark for legal tasks.
- stack
- Qwen2.5
- QLoRA
- LoRA
- Unsloth
- LegalBench
- legal-qa-v1
10 / 15
tasks where QLoRA beats the base 3B
several from 0.0 to 0.4-0.7
11 / 15
tasks where QLoRA beats standard LoRA
r=4, 4-bit vs r=16, full precision
8 GB
the entire GPU budget (laptop RTX 4060)
model picked to fit it
● base 3B · ● QLoRA · ● LoRA · ◦ base 7B · hover a row to read
Picking a model that fits 8 GB
On vals.ai's legal leaderboard, Qwen2.5 Turbo Instruct (7B) is the top open-weights model that loads on a laptop 4060 with 4-bit quantization. It outscores far larger models there, Llama 2 70B and Gemma 2 27B included. QLoRA fine-tuning fits the 7B on that card; full LoRA does not, so the adapters train on the 3B and the 7B stays in the benchmark as a reference.
Both adapters trained with Unsloth on the legal-qa-v1 dataset. Scoring follows LegalBench's protocol: few-shot prompts, under 10 examples per task, across the 15 tasks the dataset covers.
QLoRA rank 4 against LoRA rank 16
QLoRA at rank 4 with 4-bit quantization beats standard LoRA at rank 16 on 11 of 15 tasks, despite the higher training loss (2.1034 against 2.0616). The extra capacity let LoRA fit the training set more closely without transferring to the held-out tasks. On contract tasks the gap is wide: ucc_v_common_law goes from 0.0 (base) to 0.70 (QLoRA), and contract_qa from 0.0 to 0.63.
The base 7B still leads on 6 tasks of general legal reasoning (personal_jurisdiction, proa, consumer_contracts_qa among them); the adapters win the specialized contract-interpretation tasks. The adapters add accuracy on the contract tasks where the base scores near zero, at 3B scale on a laptop GPU.