VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
Easy10 XPattention

Why is long-context attention expensive?

How does the compute of standard self-attention grow with sequence length n?

Stuck? Show the solution

Answer: C. O(n²): every token attends to every other token

The attention matrix has n × n entries, so doubling the context roughly quadruples attention compute and memory.

Learn the idea behind itThe Transformer I — Attention, Positions and Heads from First Principles