VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D13: GPU & CUDA Programming · The GPU Memory Hierarchy — Coalescing, Shared Memory and Bank Conflicts

The GPU Memory Hierarchy — Coalescing, Shared Memory and Bank Conflicts

15 min read·Staff+ Tier· Unrolled Pro
The GPU Memory Hierarchy — Coalescing, Shared Memory and Bank Conflicts

You already know: cache lines and locality on CPUs (D1 C++ & memory); registers, shared memory, L2 and HBM sizes (GPU architecture); thread indexing and the naive transpose that reads well and writes badly (CUDA programming model); FlashAttention's tiles in SRAM (D12 Advanced optimization).

Why this matters for Atlas. Most kernels in an LLM are memory-bound, so the memory system decides their speed. Three rules cover most of it: make a warp's accesses to global memory coalesced (adjacent threads, adjacent addresses), reuse data from fast on-chip memory (registers and shared memory) instead of re-reading HBM, and lay out shared memory to avoid bank conflicts. This lesson quantifies each rule with simulations of the hardware's actual behaviour and builds the tiled transpose that fixes the previous lesson's slow writes.


1. The hierarchy, and what each level is for

The question. Where should a kernel keep its data?

Start from nothing. Each level trades size for speed and scope:

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "The GPU Memory Hierarchy — Coalescing, Shared Memory and Bank Conflicts"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee