You already know: contrastive learning and the role of negatives (Vision-language models); diffusion and VAEs (Generative image models); robots, control loops and sim-to-real (Spatial AI & robotics); reading and reproducing research claims (this domain).
Why this matters for Atlas. Two ideas are at the centre of current debates about what comes after scaling next-token prediction. A world model learns how an environment evolves (given state and action, what happens next), so an agent can plan by imagining outcomes instead of trying them for real. Joint-embedding predictive architectures (JEPA) learn representations by predicting the embedding of missing or future content rather than its raw pixels or tokens, ignoring unpredictable detail. This lesson builds both from first principles: why latent prediction collapses and how to prevent it, and how planning with a learned model works, so you can read this fast-moving literature critically.
1. What to predict: pixels, tokens or embeddings
The question. Self-supervised learning predicts part of the data from another part. Which part, and in what space?
Start from nothing. Three families: