Problem. Given queries Q (B × T_q × d), keys K (B × T_k × d), values V (B × T_k × d_v) and a boolean mask saying which
(query, key) pairs may attend, compute
scores = Q Kᵀ / √d (B × T_q × T_k)
scores[masked] = −∞
weights = softmax(scores, over keys)
out = weights V (B × T_q × d_v)
with two standard masks: a padding mask (keys that are padding, from lesson 15.3's batch mask) and a causal mask (query i may only see keys j ≤ i). Return the output and the weights.
Three details decide whether this works: the softmax must be stable (lesson 15.6), masked entries must get zero weight, and a query whose keys are all masked must not produce NaN (−∞ − (−∞) is NaN). The 1/√d scale keeps scores in a range where softmax still has useful gradients.
ML context. This is the core of every transformer layer. In an ML round it's usually asked as "implement attention with a mask", with follow-ups about multi-head reshaping, memory for long sequences (the T × T matrix), and the KV cache used for fast generation. All three are covered here.