Consider $$f(x)=\tfrac12x^THx-b^Tx+c,$$ where $H$ is symmetric. Its gradient is $\nabla f(x)=Hx-b$. If $H$ is positive definite, the objective has one minimizer $x^*=H^{-1}b$. Quadratic functions are simple enough for exact analysis and approximate smooth objectives near a local point.
Let invariant directions of $H$ satisfy $Hv_i=\lambda_iv_i$. Along coordinate $z_i=v_i^Tx$, curvature is $\lambda_i$. Large values create steep directions and small positive values create shallow directions. Positive definiteness means every $\lambda_i>0$, so the surface bends upward in all directions.
Gradient descent gives $x_{t+1}=x_t-\alpha(Hx_t-b)$. Subtracting the optimum and expressing error along $v_i$ yields $$e_{t+1,i}=(1-\alpha\lambda_i)e_{t,i}.$$ Stability in every direction requires $|1-\alpha\lambda_i|<1$, so a constant step must satisfy $0<\alpha<2/\lambda_{max}$.
The condition number is $\kappa=\lambda_{max}/\lambda_{min}$. When it is large, the learning rate must respect the steepest direction while progress along the shallowest direction remains slow. Geometrically, level sets become elongated ellipses and updates can zigzag across the narrow axis.
Preconditioning applies a matrix $P$ to the gradient: $x_{t+1}=x_t-\alpha P(Hx_t-b)$. An ideal $P=H^{-1}$ equalizes curvature and reaches the quadratic optimum in one unit step, but forming the exact inverse is usually too expensive. Diagonal or structured approximations seek part of this benefit at lower cost.
Neural objectives are not globally quadratic, yet local curvature still explains sensitivity to learning rate and parameter scaling. Track update-to-parameter ratios and loss response to small steps. Rescaling features, normalizing activations, or changing parameterization can improve conditioning without changing the predictions representable by the model.