VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D13: GPU & CUDA Programming · GPU Architecture — Thousands of Simple Cores and a Memory System to Feed Them

GPU Architecture — Thousands of Simple Cores and a Memory System to Feed Them

15 min read·Staff+ Tier· Unrolled Pro
GPU Architecture — Thousands of Simple Cores and a Memory System to Feed Them

You already know: caches, cache lines and why memory layout decides speed (D1 C++ & memory); processes, threads and context switches (D1 Operating systems); the roofline, arithmetic intensity and memory-bound decode (D12 Inference fundamentals); the D12 bridge's fused-kernel example.

Why this matters for Atlas. Every model in this course runs on GPUs, and every performance number in Domain 12 came from two datasheet figures. Reaching them depends on how the chip is built: many simple cores grouped into streaming multiprocessors (SMs), threads executed 32 at a time in warps, tensor cores that only run matrix tiles in specific formats, and a memory system whose bandwidth you get only with the right access patterns. This lesson explains the architecture from first principles and quantifies the ideas the rest of the domain relies on: latency hiding, SIMT execution and divergence, and the difference between CUDA cores and tensor cores.


1. Latency machines and throughput machines

The question. A CPU core is much faster than a GPU core. Why do GPUs win for deep learning?

Start from nothing. A CPU is optimized for latency: make one thread finish as fast as possible. It spends its transistors on large caches, branch prediction, out-of-order execution and high clock speeds, so a handful of cores each run a complex instruction stream quickly. A GPU is optimized for throughput: finish as much total work as possible. It spends transistors on many simple arithmetic units and on keeping thousands of threads in flight, and hides each thread's slowness by switching to another one that's ready.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "GPU Architecture — Thousands of Simple Cores and a Memory System to Feed Them"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee