LoRA and Parameter-Efficient Fine-tuning

Full fine-tuning updates every weight in a model, which is expensive to train and produces a full-size checkpoint per task. LoRA freezes the base model and trains small low-rank adapter matrices instead, giving tiny swappable checkpoints; QLoRA adds a quantized frozen base so the whole thing fits on a single GPU. FDE loops probe it because it is how you adapt a model on a customer's data without their budget or their hardware blowing up.