You already know: probability distributions and the Gaussian (D2); training networks with gradient descent (D4); encoders and embeddings (D8, Vision-language models); text–image embeddings (CLIP).
Why this matters for Atlas. Generating an image means sampling from the distribution of plausible images, a distribution over millions of pixel values. Three ideas made that tractable: autoencoders compress images into a small latent space; variational autoencoders make that space smooth enough to sample from; diffusion models learn to undo noise one small step at a time, which turns one impossible jump (noise → image) into many easy ones. Modern text-to-image systems run diffusion in an autoencoder's latent space, conditioned on text embeddings. This lesson builds each idea in NumPy, then covers guidance, cost, and the safety issues of generated media.
1. Autoencoders: compressing to a latent space
The question. Most of the numbers in an image are redundant. How do you find the few that matter?
Start from nothing. An autoencoder has an encoder mapping the input x to a small code z, and a decoder mapping z back to x̂, trained to minimize reconstruction error ‖x − x̂‖². The bottleneck forces z to keep only what's needed to rebuild x. A linear autoencoder with squared error learns the same subspace as PCA (D2); nonlinear ones learn curved manifolds. Latent diffusion's autoencoder compresses a 512×512×3 image by about 48× into a 64×64×4 latent, so the expensive generative model works on far fewer numbers.