For vectors $x,y\in\mathbb{R}^d$, the dot product is $x^Ty=\sum_{i=1}^{d}x_iy_i$ and the L2 norm is $\lVert x\rVert_2=\sqrt{x^Tx}$. The dot product mixes alignment and magnitude. Two vectors can have a large dot product because they point similarly, because they are long, or both.
Cosine similarity removes magnitude: $$\cos(x,y)=\frac{x^Ty}{\lVert x\rVert_2\lVert y\rVert_2}.$$ For nonzero vectors it lies between negative one and one. A value near one means similar direction, zero means orthogonality, and negative one means opposite direction. Near-zero norms require explicit numerical handling.
Let $u$ be a unit vector. The projection of $x$ onto the line spanned by $u$ is $$\operatorname{proj}_u(x)=(u^Tx)u.$$ The scalar $u^Tx$ is the coordinate along that direction. The residual $r=x-(u^Tx)u$ satisfies $u^Tr=0$, so projection and residual are orthogonal components.
For a matrix $U$ whose columns form an orthonormal basis of a subspace, projection is $\hat x=UU^Tx$. If columns are independent but not orthonormal, use $$\hat x=U(U^TU)^{-1}U^Tx.$$ The inverse corrects for overlap and scale among basis vectors. Poorly conditioned columns make this computation sensitive to numerical error.
Projection solves a least-squares problem: $\hat x$ is the point in the chosen subspace minimizing $\lVert x-z\rVert_2^2$. In representation learning, a linear probe, low-dimensional feature subspace, or retrieval direction often asks how much of an embedding lies along selected directions. The result depends entirely on the representation's coordinate geometry.
Normalize embeddings only when magnitude should not carry information. Cosine similarity makes $x$ and $cx$ identical for positive $c$, which is useful for direction-based retrieval but harmful if norm encodes certainty or frequency. Log norm distributions, guard against zero vectors, and compare cosine and dot-product rankings before choosing a metric.