You already know: caches, cache lines and why memory layout decides speed (D1 C++ & memory); processes, threads and context switches (D1 Operating systems); the roofline, arithmetic intensity and memory-bound decode (D12 Inference fundamentals); the D12 bridge's fused-kernel example.
Why this matters for Atlas. Every model in this course runs on GPUs, and every performance number in Domain 12 came from two datasheet figures. Reaching them depends on how the chip is built: many simple cores grouped into streaming multiprocessors (SMs), threads executed 32 at a time in warps, tensor cores that only run matrix tiles in specific formats, and a memory system whose bandwidth you get only with the right access patterns. This lesson explains the architecture from first principles and quantifies the ideas the rest of the domain relies on: latency hiding, SIMT execution and divergence, and the difference between CUDA cores and tensor cores.
1. Latency machines and throughput machines
The question. A CPU core is much faster than a GPU core. Why do GPUs win for deep learning?
Start from nothing. A CPU is optimized for latency: make one thread finish as fast as possible. It spends its transistors on large caches, branch prediction, out-of-order execution and high clock speeds, so a handful of cores each run a complex instruction stream quickly. A GPU is optimized for throughput: finish as much total work as possible. It spends transistors on many simple arithmetic units and on keeping thousands of threads in flight, and hides each thread's slowness by switching to another one that's ready.