Skip to content

Course Map

Every chapter of this course is a reply to an earlier one. The map below puts all of them on one timeline, from the McCulloch–Pitts neuron of 1943 to the world models of 2026, together with the approaches outside the course that the chapters stand on: LSTMs, normalizing flows, contrastive learning, MuZero and a few more.

The map

Click a box to follow its lineage.
The topics of the course and related approaches on a timeline, from 1943 to 2026, in six columns, with arrows from each idea to the ones built on it.
Each box is an idea, placed by year; time runs downward. Solid boxes are chapters of the course, dashed ones are approaches outside it. Click a box: its ancestors light up in orange, its descendants in green, and the caption explains the idea in plain words, with a link to its paper.

How to read it:

  • Rows are time. Each box sits at its year, and time runs downward. The scale is compressed before 2005 and stretched after it, where most of the course lives. Boxes of the same year in the same column stack, so a box can sit a little below its year's line; the year printed in the box is the one that counts.
  • Columns are themes: data and evaluation, learning and training, architectures, language and LLMs, generative models, representations and world models.
  • Solid boxes are chapters of the course, with the chapter number. Dashed boxes are outside it. Course chapters only dims the dashed ones.
  • Arrows are lineage: an idea points to the ones built on it. Click a box to light its ancestors in orange and its descendants in green; the caption links to the chapter.
  • Click a box to read it in plain words. Below the map, each idea gets two short paragraphs: what it contributed, explained as you would to someone new to the field, and where it is used today, followed by a link to the paper that introduced it.

Five threads worth following

Click the last box of each thread and the map draws it for you.

  1. The training stack. McCulloch–Pitts → perceptron → backpropagation, with Robbins–Monro's stochastic approximation underneath. Every later box inherits it: initialization, normalization, dropout and Adam are what made depth trainable, and ResNet's identity path is what made it deep.
  2. From convolutions to patches. Neocognitron → LeNet → AlexNet → ResNet, then the Transformer takes over vision with ViT. The same patches feed MAE, CLIP, DiT and I-JEPA.
  3. From word vectors to reasoning models. A neural language model (2003) → word2vec → seq2seq → attention → Transformer → GPT → scaling laws → GPT-3 → RLHF → test-time compute. The evaluation column follows: perplexity first, an LLM as judge last.
  4. The generative family tree. The autoencoder → VAE → VQ-VAE → VQGAN → latent diffusion → DiT. Diffusion arrives from its own root (2015) through score models and DDPM; flow matching arrives from normalizing flows through neural ODEs; both meet in SD3 and FLUX. Autoregressive image models take the other branch of VQGAN, the tokens.
  5. Predicting representations instead of pixels. CPC → SimCLR → BYOL → JEPA → I-JEPA → LeJEPA, while world models come down from Dyna and the VAE through PlaNet and Dreamer. The two lines meet in LeWorldModel and TDWM.

What the map leaves out

A map with one box per idea has to choose. Hardware (GPUs, TPUs), datasets other than ImageNet and most engineering (mixed precision, distributed training, FlashAttention) are left out, as are many parallel inventions. A year is the year of the paper that introduced the idea, usually the arXiv version; where the course quotes a different year, the box follows the course.