An attention mask defines which key positions each query may use. Masks enforce the structure of a task. They are not learned preferences: a forbidden connection should contribute zero attention regardless of its raw similarity score.
A causal mask blocks future positions during next-token training. Position three may see positions zero through three but not four or later. This prevents answer leakage and makes parallel training behave like left-to-right generation.
A padding mask blocks placeholder tokens added to equalize batch lengths. Without it, real tokens can attend to meaningless padding. Queries that are themselves padding should also be excluded from losses and downstream metrics.
Masks are usually applied to scores before softmax by adding a very negative value at forbidden entries. Setting weights to zero after softmax is wrong unless the remaining weights are normalized again, because the row would no longer sum to one.
Broadcasting can silently apply a mask along the wrong dimension. Write expected shapes for batch, heads, query positions, and key positions. Test with different sequence lengths so a coincidental square shape does not hide the error.
Create unit tests that assert every future and padded weight is effectively zero, every valid row sums to one, and no fully masked row creates invalid numbers. Visualize a tiny mask matrix before debugging a full model.