A computation graph breaks a complicated function into simple operations. Suppose scalar loss $L$ depends on intermediate values $u_1,\ldots,u_m$, which ultimately depend on parameters $\theta$. The chain rule says a parameter influences the loss through every directed path connecting that parameter to $L$.
For an intermediate scalar $v$, define its adjoint $\bar v=\partial L/\partial v$. If operation $z=f(x,y)$ receives upstream adjoint $\bar z$, it contributes $$\bar x\mathrel{+}=\bar z\frac{\partial z}{\partial x},\qquad \bar y\mathrel{+}=\bar z\frac{\partial z}{\partial y}.$$ The plus-equals matters because one value may influence the loss through several downstream paths.
Reverse mode begins with $\bar L=1$ and visits operations in reverse topological order. Each stored forward value supplies the local derivative needed for its backward rule. Once every contribution reaches a parameter, its adjoint is exactly that parameter's loss gradient.
For affine map $y=Wx+b$ and upstream vector $\bar y=\partial L/\partial y$, the reverse rules are $$\bar x=W^T\bar y,\qquad \bar W=\bar yx^T,\qquad \bar b=\bar y.$$ These shapes provide a strong check: each gradient has the same shape as the variable it differentiates.
Reverse mode is efficient when outputs are few and inputs are many. One backward sweep computes every component of $\nabla_\theta L$ at cost comparable to a small multiple of the forward computation. Forward mode instead propagates sensitivity from selected inputs and is attractive when there are few inputs and many outputs.
Automatic differentiation applies exact local rules to the executed program, but it does not guarantee that the program represents the intended mathematics. Test custom operations with finite differences, check nondifferentiable points, and inspect gradient shapes and scales. Detaching a value deliberately cuts paths; detaching accidentally produces a plausible forward result with missing gradients.