The ML Landscape

Before diving into algorithms, we need a map: what kinds of learning exist, what a real ML project looks like from end to end, and what it means for a model to generalize.

The lesson follows that map in four stages, drawn below: 1. kinds of learning, sorted by the feedback the learner gets; 2. the project loop, the steps of a real project from the question to a deployed model; 3. generalization, what makes a model good on data it has never seen; 4. responsibility, the questions to ask before a model decides about people. Each stage links to its section, and in the slides a strip at the bottom shows which stage you are in.

Slides

Click the slides and use the ← → arrows to navigate; F goes full screen and Esc shows every slide. Open the slides in a new tab

Learning paradigms

Stage 1. Machine learning problems are classified by the kind of feedback available to the learner.

Supervised learning

The dataset contains inputs \(x\) and the desired outputs \(y\) (labels). The goal is to learn a function \(f\) such that \(f(x) \approx y\) on new data.

• Regression — \(y\) is continuous: predicting house prices, energy demand, a patient's length of stay (the regression lessons).
• Classification — \(y\) is categorical: spam/ham, tumor benign/malignant, which digit is in the image (the classification lessons, trees and ensembles).

Unsupervised learning

Only inputs \(x\) — no labels. The goal is to discover structure:

• Clustering — group similar observations, e.g. customer segments (Clustering);
• Dimensionality reduction — compress many features into few informative ones (PCA, t-SNE, UMAP);
• Topic modeling — discover themes in a collection of documents (BERTopic).

Reinforcement learning

An agent interacts with an environment, receives rewards, and learns a policy that maximizes long-term reward — the paradigm behind game-playing systems (AlphaGo) and robotic control. It is out of scope for this course, but you should recognize it on the map.

The ML workflow

Stage 2. A model is a small part of a larger, iterative process. This course is organized around this loop:

Two practical truths about this diagram:

1. Most of the work is in steps 2–3. Practitioners routinely report spending the majority of their time understanding and preparing data, not fitting models.
2. The loop never ends. Deployed models decay as the world changes (drift); monitoring and retraining are part of the job, not an afterthought.

Generalization: the central problem

Stage 3. A model that memorizes its training data perfectly can still be useless. What matters is performance on data it has never seen.

• Underfitting: the model is too simple to capture the pattern — poor performance even on training data.
• Overfitting: the model captures noise as if it were signal — excellent on training data, poor on new data.

In the simulator below, move the polynomial degree from 1 to 15 and watch both errors: where is the model underfitting, and where does it start overfitting? Then press more data and slide the degree up again.

The training error only goes down as the model gets more flexible; the test error is what tells you when it has started memorizing noise. With more data the same flexible model has much less room to overfit.

Everything in the lessons on regularization, cross-validation and model selection — train/test splits, cross-validation, regularization, the bias–variance trade-off — exists to manage this tension. For now, keep one rule:

No free lunch

The No Free Lunch theorem (Wolpert, 1996) says that averaged over all possible problems, no learning algorithm is better than any other. In practice this means: there is no universally best model — you must try several families and validate. That is why this course teaches a portfolio (linear models, neighbors, kernels, trees, ensembles, networks) rather than a single silver bullet.

Ethics and responsibility

Stage 4. Models trained on historical data inherit historical bias. Before shipping a model, ask:

• Fairness — does the model perform equally across demographic groups? A credit model trained on biased decisions reproduces them at scale.
• Privacy — was the data collected with consent? Can individuals be re-identified?
• Transparency — can decisions be explained to those affected? (The explainability lesson covers it.)
• Feedback loops — does deploying the model change the data it will be retrained on? (Predictive policing is the canonical cautionary tale.)

References

• Wolpert, D. H. "The Lack of A Priori Distinctions Between Learning Algorithms." Neural Computation 8 (1996). DOI

Quiz