Two operations are often described as weight decay. The first adds $\tfrac{\lambda}{2}\lVert\theta\rVert_2^2$ to the objective, producing the gradient $g_t+\lambda\theta_{t-1}$. The second directly shrinks parameters during the update. These operations can match under plain gradient descent with a suitable convention, but they are generally different under coordinate-wise adaptive scaling.
Let $P_t$ denote a diagonal adaptive preconditioner. Coupled L2 regularization gives $\theta_t=\theta_{t-1}-\alpha_tP_t(g_t+\lambda\theta_{t-1})$. The regularization contribution is therefore $-\alpha_t\lambda P_t\theta_{t-1}$. Coordinates with different entries in $P_t$ receive different amounts of shrinkage, and the penalty also enters any gradient-moment statistics used to construct $P_t$.
Decoupled decay instead uses $\theta_t=(1-\alpha_t\lambda)\theta_{t-1}-\alpha_tP_tg_t$. The data gradient alone updates the adaptive statistics, while the shrinkage acts directly on the parameters. This separation makes the meaning of $\lambda$ clearer, but it does not make $\lambda$ independent of the learning-rate schedule.
With no gradient update, repeated decoupled decay gives $\theta_T=\theta_0\prod_{t=1}^{T}(1-\alpha_t\lambda)$. When every $\alpha_t\lambda$ is small, this is approximately $\theta_T\approx\theta_0\exp(-\lambda\sum_{t=1}^{T}\alpha_t)$. Cumulative shrinkage depends on the area under the learning-rate schedule, so changing training length or schedule can require retuning decay.
Decay need not apply to every parameter group. Bias vectors, offsets, and normalization scale parameters are often treated separately because shrinking them may not express the same structural preference as shrinking a weight matrix. This is an architectural decision, not a universal law. Record the exact inclusion rules in the training configuration so a resumed or reproduced run uses the same objective.
Tune decay against validation behavior while also watching training loss and parameter norms. Excessive decay can underfit even when optimization is stable; insufficient decay can allow norms to grow without improving validation performance. Compare settings using the same optimizer, learning-rate schedule, number of updates, and parameter-group rules, because changing several of these at once obscures the cause.