VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
DSA for AI/ML · Module 15: The ML coding round

Lesson 15.7 of 12 in this module Unrolled Pro

Scaled dot-product attention with masking

Compute softmax(QKᵀ/√d)V for a batch of sequences with padding and causal masks, in NumPy. From loops over batch, query and key positions, to batched matrix products, masks applied as −∞ before a max-shifted softmax, a guard for rows where every key is masked (which would otherwise produce NaN), and the 1/√d scale that keeps softmax from saturating. Then multiple heads by reshaping, sequences too long for a T × T score matrix (chunked online softmax), a key-value cache for generation, and cross-attention.

about 15 min 3 Python examples 2 quick checks

Problem. Given queries Q (B × T_q × d), keys K (B × T_k × d), values V (B × T_k × d_v) and a boolean mask saying which (query, key) pairs may attend, compute

text
scores = Q Kᵀ / √d                       (B × T_q × T_k)
scores[masked] = −∞
weights = softmax(scores, over keys)
out = weights V                          (B × T_q × d_v)

with two standard masks: a padding mask (keys that are padding, from lesson 15.3's batch mask) and a causal mask (query i may only see keys j ≤ i). Return the output and the weights.

Three details decide whether this works: the softmax must be stable (lesson 15.6), masked entries must get zero weight, and a query whose keys are all masked must not produce NaN (−∞ − (−∞) is NaN). The 1/√d scale keeps scores in a range where softmax still has useful gradients.

ML context. This is the core of every transformer layer. In an ML round it's usually asked as "implement attention with a mask", with follow-ups about multi-head reshaping, memory for long sequences (the T × T matrix), and the KV cache used for fast generation. All three are covered here.


1. Step 1: Clarify

That's the preview. Modules 00–01 are free; this lesson is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Scaled dot-product attention with masking"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee