Interview prep Machine Learning Classes F1 to E5

Machine Learning Interviews

The 77 machine learning (ML) topics that come up most in internship and job interviews, from exploratory analysis to neural networks and ML operations (MLOps): what to study, how to study it and how to answer. With short answers, a quiz, a checklist to track your progress and, for every topic, a link to the class where it is taught.

The 2 p.m. question

A scene that is going to happen to you.

Thursday, 2 p.m. An interview for the internship you wanted most, on a data team. After the introductions, the interviewer turns her laptop toward you: a churn model (customers who cancel), trained by someone on the team, with 95% accuracy on the test set. And she asks, with nothing else:

"Is this model any good?"
She does not want a yes or a no. She wants to see how you think.

There is a weak answer ("yes, 95% is very high") and a strong one, which starts with a question back: "what is the churn rate?". If only 5% of customers cancel, a model that never predicts a cancellation already scores 95% accuracy without having learned anything.

The difference between the two answers is not memorizing more definitions. It is knowing how to connect what you have learned: baselines, class imbalance and the confusion matrix, thresholds and the cost of errors. This page lays out that path. At the end, we come back to the 2 p.m. question with the full answer.

What an ML interview looks like

Three kinds of question, three skills, one logic: reasoning about data out loud.

Conceptual

"What is overfitting?"

Tests whether you can define it in your own words, give the intuition and say when the idea breaks down. There is almost always an "it depends" hiding in it.

Case

"Training 0.99, validation 0.72"

Tests diagnosis and next steps. There is no single answer: there is structured reasoning and justified choices.

Coding

"Build this pipeline"

Tests whether you use scikit-learn without leaking: Pipeline, ColumnTransformer, cross_val_score.

What separates candidates

For an entry-level role, almost nobody expects you to know the newest algorithm. What sinks candidates are the basic traps: comparing without a baseline, reporting accuracy on imbalanced data, fitting transformations before splitting the data, choosing hyperparameters by looking at the test set. Avoid those four and you are already ahead of most.

What the interviewer is assessing

Framing the problem Choosing the metric Avoiding leakage Comparing with a baseline Diagnosing errors Stating limitations

Aspect	Conceptual	Case	Coding
Right answer	One definition, with nuances	Several valid answers	Code that runs and does not leak
Focus	Clarity and intuition	Diagnosis and trade-offs	Using the tools correctly
Typical length	1–3 min per question	10–20 min	15–30 min
Most common mistake	Reciting without the trap	Jumping to a model without asking for context	Preprocessing before the split
Where to practice	Key concepts	Case questions	The class labs, redone without notes

How to answer

8 principles for turning what you know into an answer that convinces.

1

Ask for context before answering What is the business problem? How common is the positive class? What does each kind of error cost? Is the data ordered in time? Are there several rows per customer? A well-placed question shows more maturity than a perfect definition. It is what rescued the 2 p.m. answer.

2

Define it in one sentence, in your own words "Overfitting is when the model memorizes the noise in the training data and gets new data wrong." Short, no needless jargon. If you need three sentences to define it, you do not own the concept yet.

3

Draw The U-shaped validation error curve, the 2×2 confusion matrix, the threshold ruler, the cross-validation folds, the elbow of the k-means inertia. A ten-second sketch says more than a minute of talk and organizes your own thinking.

4

Give a concrete example Ideally one you ran yourself: "in the cross-validation class, with 10 exams per patient, shuffled KFold reported 93.8% accuracy and GroupKFold 78.0%; on 200 new patients the model scored 75.45%, so the grouped estimate was the honest one". Examples you lived through prove you did not just memorize (C5).

5

Name the trap Every good answer ends with "…but be careful: …". Accuracy misleads with rare classes; R² does not tell you whether the error is acceptable; a silhouette score does not prove that clusters exist; feature importance is not causation. This is where the interviewer sees depth.

6

Speak in numbers "5% churn", "a 0.27 gap between training and validation", "the 0.004 difference is within the 0.02 spread across folds". Numbers make your reasoning checkable and show that you think like someone who has measured things.

7

Always compare with something A number on its own says nothing. Compare with the baseline, with the previous model, with the variability of cross-validation, with the same procedure on shuffled data. "0.81 against the baseline's 0.62" is an answer; "0.81" is not.

8

Lead: propose the next step Do not wait for the next question. "The next step would be to look at the errors by customer segment" or "I would try removing that feature to see whether it is leakage". It shows you know what to do next, not just what things are.

Practical tip: signpost your transitions. When you move from diagnosis to solution, say it out loud: "so the problem looks like variance; I'll suggest three ways to reduce it, from the cheapest to the most expensive". The interviewer follows your structure and you do not lose your way.

Case questions, step by step

"Walk me through how you would build a model to…" — the most common open question. The answer is the ML workflow, a loop of six steps.

1

Frame the problem What is the unit of analysis (what is one row)? What is the target? Is it classification, regression or a problem without labels (clustering, dimensionality reduction)? Which metric reflects the real cost of errors, and what value of it would count as success? When should you NOT use ML here? F2 · The ML Landscape →

2

Split the data and lock the test set away Before looking at anything: training, validation (or cross-validation) and test. A stratified split if a class is rare; a temporal one if the model will predict the future; a grouped one if there are several rows per customer or patient. Exploration happens on the training set only. D1 · Exploratory Data Analysis →

3

Build the pipeline and measure the baseline Imputation, scaling and encoding inside a Pipeline with a ColumnTransformer, fitted on the training part of each fold only. The baseline (the mean or the majority class) sets the floor. D3 · Pipelines →

4

Compare models under a single protocol The same cross-validation folds for every candidate, each one already tuned. Report the mean and the standard deviation. Prefer the simpler model when the difference is within the variability. C5 · Cross-Validation →

5

Analyze the errors, test once and state the limits Where does the model fail most (by subgroup, by range)? Which features matter according to permutation importance? Only then measure on the test set (once) and write down what the model cannot support. E2 · Explainability →

6

Deploy and monitor Save the whole pipeline together with the library versions, validate the inputs, monitor the features and the scores while the labels have not arrived, and decide when to retrain. The world changes and the model ages. E4 · MLOps →

The key scoping question

Always ask: "what does it cost to be wrong in each direction?" It changes everything. If a false negative is expensive (an undetected patient, a fraud that slips through), you will favor recall and lower the threshold. If a false positive is expensive (blocking an honest customer), you will favor precision. The metric and the threshold come from that answer, not from the default of predict.

Quick diagnosis

The symptoms that show up in case questions, what they usually point to and what to do about them.

In case questions, the interviewer describes a symptom and expects a diagnosis with a next step. Before proposing a new model, rule out the cheap causes: leakage, a poorly chosen baseline, the wrong metric, feature scales, different distributions in training and validation.

Symptom	Likely diagnosis	Next step
Training 0.99, validation 0.72	Overfitting (high variance)	Rule out leakage; reduce complexity, regularize, get more data
Training and validation both poor	Underfitting (high bias)	A more flexible model or better features; more data will not help
95% accuracy on a rare-event problem	Possibly just the majority class	Compare with the baseline; look at recall, precision, PR-AUC (area under the precision–recall curve)
One feature alone gives an AUC (area under the ROC, receiver operating characteristic, curve) of 0.99	Target leakage	Check when the feature is recorded; remove it and measure the drop
Model B beats A by 0.004	A difference within the noise	Look at the spread across folds; prefer the simpler one
Great on validation, much worse on test	Overfitting the validation set through tuning, or a leaky split	Nested CV (nested cross-validation); a temporal or grouped split; a test set truly locked away
Coefficients with absurd signs	Multicollinearity	Do not read them one by one; regularize (ridge); drop redundant features
U-shaped pattern in the residuals	A nonlinear relationship the model misses	Transform features, add terms, use a nonlinear model
The k-means clusters follow a single column	Features on different scales: the one with the largest unit dominates the distance	Standardize inside the pipeline; decide scaling from the units, not from the silhouette
The training loss oscillates or blows up	Learning rate too large (or unscaled features)	Lower the learning rate; standardize the inputs
Great on validation, poor from day one in production	Training–serving skew, or a feature that does not exist at prediction time	Compare how features are computed in training and in serving; validate inputs against a schema
In production, slowly gets worse with no code change	Drift: the inputs or their relationship with the target changed	Monitor features and scores (PSI, the population stability index; the Kolmogorov–Smirnov or KS test); measure when the labels arrive; retrain on recent data

A complete example

"Training 0.99, validation 0.72"

Ask: which model, how much data, how was the split done?

Rule out the cheap causes: does any feature exist only after the target? Are rows from the same customer on both sides? Do training and validation come from the same distribution?

Diagnose: a 0.27 gap means high variance: the model memorized the training set.

Propose, cheapest first: limit the depth or regularize; drop noisy features; collect more data, if the learning curve shows it would help. Show a validation curve to choose the complexity.

Tools by symptom

What to use when

Validation curve: choose the complexity (depth, regularization) from the U of the error.

Learning curve: find out whether more data is worth it.

Cross-validation: estimate performance and its variability with less split luck.

Confusion matrix: see what kind of errors the model makes, not just how many.

Permutation importance and SHAP (SHapley Additive exPlanations): learn which features the model uses on new data, and for each prediction.

Shuffled data: the yardstick for any clustering and any suspicious score.

PSI and the KS test: detect that the inputs have changed in production.

The 77 key concepts

The 77 topics, in eight themes. Try answering out loud before opening each theme. The numbers identify each topic in the checklist; the code next to each question takes you to the class where the topic is taught.

[1]Supervised vs unsupervised?F2

In supervised learning every example comes with the right answer (the label $y$) and the model learns to predict it from the features $X$. In unsupervised learning there is only $X$ and the goal is to find structure: groups, main directions, topics. There is also reinforcement learning, where an agent learns by trial and reward. Trap: without labels there is no obvious accuracy metric, and evaluation gets harder.

[2]Classification vs regression?F2

It depends on the target: a category (fraud or not) is classification; a continuous number (a price) is regression. Trap: some problems fit both (a 1-to-5 rating); the choice comes from how errors cost and whether the order matters.

[3]What is a feature?D1

An input variable used to predict the target. First define the unit of analysis: what one row is (a customer, a transaction, a day). Trap: a feature is only valid if it is available at prediction time.

[4]Why split into training and test sets?C5

To estimate performance on data the model has never seen. Scoring on the training data rewards memorization. The test set stays locked until the final measurement; choosing between models is what validation is for. Trap: looking at the test set many times turns it into a validation set.

[5]What is overfitting?F2

The model learned the noise in the training data along with the pattern: low training error and high test error. It happens with models too complex for the amount of data. Trap: the signal is the gap between training and validation, not a high training score.

[6]What is underfitting?M1

The model is too simple for the pattern: high error on training and on test. More data will not fix it; you need a more flexible model or better features.

[7]What is a baseline?C1

A trivial model, such as always predicting the mean (regression) or the majority class (classification): DummyRegressor and DummyClassifier. It sets the floor: a model that does not beat the baseline is useless, however sophisticated it is.

[8]Describe the workflow of an ML project.F2

Frame (target, unit of analysis, metric, success criterion) → split and lock the test set → explore the training set only → preprocessing pipeline → baseline → compare models with cross-validation → analyze errors → test once → deploy and monitor. It is a loop, not a line: evaluation sends you back to data preparation and drift sends you back to the data. See the diagram above.

In class: F2 · The ML Landscape · D1 · Exploratory Data Analysis · C5 · Cross-Validation & Data Leakage · M1 · Model Selection

[9]What is data leakage?C5

Information that would not be available at prediction time gets into training or evaluation, and the measured performance looks better than it will be in practice. It is the most common cause of models that "worked in the notebook".

[10]What is target leakage?C5

A feature that is a consequence of the target or is recorded after it, such as using "received antibiotics" to predict "had an infection". Red flag: a feature with enormous importance and performance too good to be true.

[11]Why fit preprocessing on the training set only?D3

Means, standard deviations, categories and imputation values are estimated from data; if the test set takes part, its information leaks into the model. A Pipeline guarantees that fit sees only the training part of each fold, and applies exactly the same steps at prediction time.

[12]Normalization or standardization?D2

Normalization (min-max) maps each feature to $[0, 1]$; standardization gives mean 0 and standard deviation 1. It matters for models built on distances, gradients or penalties (k-NN, k-nearest neighbors; k-means; logistic regression; SVM, support vector machines; neural networks), not for trees. Trap: min-max is sensitive to outliers.

[13]What is one-hot encoding?D2

One binary column per category, without imposing an order that does not exist (integer codes invent distances). Trap: categories that only appear in the test set break the transformation; use handle_unknown="ignore".

[14]How do you handle missing values?D2

First ask why they are missing: at random, because of another observed variable, or because of the value itself. Then: drop, impute (median, mode, or from the other columns) inside the pipeline, and add an indicator when the missingness itself carries information. Trap: filling with a single value shrinks the variance.

[15]What is feature engineering?D3

Building features from the raw ones using knowledge of the problem: ratios, parts of dates, group aggregates, a title extracted from a name. It often pays more than switching models. Inside a pipeline it becomes a FunctionTransformer or a transformer of your own. Trap: aggregates computed with the test set are leakage.

[16]Why are many categories a problem?D2

One-hot creates one column per category: with thousands of them the table becomes huge and sparse, and rare categories have too few examples for the model to learn anything but noise. Alternatives: group the rare ones, encode by frequency or by the target mean. Trap: target encoding leaks if it is computed outside the folds.

[17]When do you need a temporal split?C5

When the data is ordered in time and the model will predict the future: train on the past, validate on the period that follows (TimeSeriesSplit). A random split lets the model "see the future": in class, a series that cannot be predicted by construction (a random walk) scored R² = 0.99 with shuffled KFold and −1.78 with TimeSeriesSplit.

[18]What is a stratified split?C5

A split that keeps the class proportions in every part (StratifiedKFold, stratify=y). Essential with rare classes or small samples.

[19]Several rows per patient: what do you watch out for?C5

Keep all the rows of one entity (patient, customer, store) in the same fold, with GroupKFold. Otherwise the model recognizes the patient, not the disease: in class, shuffled KFold reported 93.8% and GroupKFold 78.0%, close to the 75.45% on new patients. Guiding question: what unit will the model predict on in production? That is the unit of the split.

[20]How do you handle outliers?D1

Find out whether it is an error (fix or remove it) or a rare but real value (keep it). IQR (interquartile range) and z-score rules only flag candidates. When the extremes are real, prefer transformations (log), robust metrics (mean absolute error, MAE) or insensitive models (trees). Decide by looking at the training set only.

In class: D1 · Exploratory Data Analysis · D2 · Data Preprocessing · D3 · Pipelines · C5 · Cross-Validation & Data Leakage

[21]How does k-means work?U2

Pick $k$ centers at random; assign every point to its nearest center; move each center to the mean of its points; repeat until nothing changes. It minimizes the inertia, the sum of squared distances from each point to its center. Trap: the result depends on the initialization (use n_init well above 1) and on the column scales; and it always returns $k$ clusters, even on pure noise.

[22]How do you choose the number of clusters?U2

With the elbow of the inertia and the silhouette, which compares, for each point, the mean distance to its own cluster ($a$) with the mean distance to the nearest other cluster ($b$): $s = (b - a)/\max(a, b)$, from −1 to 1. Trap: in class, pure noise reached a silhouette of 0.40, and on the penguins it preferred 2 clusters where there are 3 species. Compare with shuffled data and with what a cluster means for the problem.

[23]When does k-means fail, and what do you use instead?U2

It assumes convex, roughly round clusters of similar spread, no noise, and a known $k$. With moons, rings, different densities or outliers, use DBSCAN (density-based spatial clustering of applications with noise) or its hierarchical version, HDBSCAN (a cluster is a dense region, and noise gets the label −1) or hierarchical clustering. Trap: DBSCAN and hierarchical clustering have no predict for new points.

[24]What is a dendrogram? And the linkage?U3

Agglomerative hierarchical clustering merges the two closest groups at every step; the dendrogram records the order and height of the merges, and cutting it at a height gives the clusters. The linkage defines the distance between groups: single linkage follows chains of points, Ward finds round groups. Trap: memory grows as $n^2$; it does not scale to millions of rows.

[25]What is the curse of dimensionality?U1

In many dimensions every pair of points ends up at almost the same distance: "nearest neighbor" loses its meaning, which hurts k-NN, k-means and any distance-based method. The culprit is independent, irrelevant directions, not the number of columns as such: data with few real dimensions written in many columns barely suffers.

[26]What does PCA (principal component analysis) do?U1

It finds the orthogonal directions of largest variance and projects the data onto the first $k$; the cumulative explained variance helps choose $k$. Traps: standardize when units differ; fit on the training set only; PCA never looks at the target, so nothing guarantees the signal lives in the first components.

[27]PCA, t-SNE or UMAP?U1

PCA is linear and fast, has a transform for new data, and preserves the global structure. t-SNE (t-distributed stochastic neighbor embedding) and UMAP (uniform manifold approximation and projection) are nonlinear and preserve neighborhoods: great for visualization. Trap: on a t-SNE or UMAP map, cluster sizes, the distances between clusters and the axes mean nothing; the map is a hypothesis, not a result, and it is not a feature for a classifier.

[28]Bag-of-words and TF-IDF?U4.2

Bag-of-words represents each document by the count of each word of the vocabulary (learned in fit). TF-IDF (term frequency–inverse document frequency) multiplies the count by a weight that falls for words present in many documents: specific words count more. Trap: word order is lost ("the dog bit the man" and "the man bit the dog" become the same vector) and the matrix is huge and sparse.

[29]Why cosine similarity for text?U4

Cosine measures the angle between vectors, not their length: a long and a short document on the same subject end up close. With rows normalized to norm 1 (the TfidfVectorizer default), cosine is just the dot product.

[30]Sparse representations or embeddings?U4 U4.3

In TF-IDF every word is its own axis: synonyms have nothing in common (in class, "o carro é rápido" and "o automóvel é veloz", "the car is fast" and "the automobile is quick", have cosine 0.00). Embeddings (word2vec, sentence embeddings) learn dense vectors in which similar meanings sit close together. Trap: TF-IDF still wins on rare, exact terms (codes, names); and a word embedding gives one vector per word, whatever the sense.

[31]What is topic modeling?U5 U5.3

Discovering the subjects of a collection of documents without labels. In LDA (latent Dirichlet allocation) and NMF (non-negative matrix factorization), a topic is a distribution over words and each document is a mixture of topics. In BERTopic, a topic is a cluster of documents: embeddings → UMAP → HDBSCAN → each cluster's words by a class-based TF-IDF (c-TF-IDF); documents in no cluster go to topic −1. Trap: topics depend on choices (stop words, number of topics) and need a human to read them.

In class: U1 · Dimensionality Reduction · U2 · Clustering · U3 · Hierarchical Clustering · U4 · Text Representation · U5 · Topic Modeling & BERTopic

In the formulas, $y$ is the true value and $\hat y$ the prediction; SSres is the sum of the squared residuals $y - \hat y$ and SStot the sum of the squares of $y$ around its mean.

MAE

mean |y − ŷ|

Mean absolute error, in the target's unit. Robust to outliers.

RMSE

√ mean (y − ŷ)²

Root mean squared error: same unit, but punishes large errors. Always ≥ MAE.

R²

1 − SSres/SStot

Share of the variance explained. 0 = predicting the mean.

[32]Assumptions of linear regression?R1

A linear relationship, independent errors, constant error variance and, for inference, roughly normal errors. Check them in the residuals: a pattern means a systematic error. Trap: for prediction, violating the assumptions matters less than for interpreting coefficients.

[33]What is MAE?R1

The mean of $|y - \hat y|$, where $y$ is the true value and $\hat y$ the prediction. In the target's unit ("on average we are off by $30k") and not very sensitive to outliers.

[34]What is RMSE?R1

$\sqrt{\tfrac{1}{n}\sum (y - \hat y)^2}$, over $n$ examples. Same unit, but it penalizes large errors more. A big gap between RMSE and MAE points to a few very large errors.

[35]What is R²?R1

$R^2 = 1 - \text{SS}_\text{res}/\text{SS}_\text{tot}$: the sum of squared residuals over the sum of squares around the mean. Zero means "as good as predicting the mean"; on the test set it can be negative. Trap: a high R² does not say whether the error is acceptable in the problem's unit; report MAE or RMSE too.

[36]What problems does multicollinearity cause?R1

With highly correlated features the model cannot separate the effect of each one: the coefficients become unstable and can take absurd signs. Interpretation suffers; prediction, hardly at all. Ridge stabilizes them by sharing the weight among the correlated features.

[37]How does gradient descent work?R2

Start from any weights and step against the gradient of the loss, with a step size set by the learning rate $\eta$. Too small and it converges slowly; too large and it oscillates or diverges. Batch uses all the data per step; stochastic, one example; mini-batch, a batch (the modern default). Trap: with uncentered features or features on different scales, the contours of the loss become a long, narrow valley: the gradient points across it, the path zigzags and the descent is slow. Standardizing the features (inside the pipeline) rounds the valley off; the closed-form solution does not care.

[38]Ridge (L2) vs lasso (L1)?R2

Both add to the loss a penalty on the size of the coefficients, with strength $\alpha$: ridge, the sum of squares; lasso, the sum of absolute values. Lasso sets coefficients to exactly zero (it selects features); ridge only shrinks them and shares the weight among correlated features. Trap: a coefficient's size depends on the feature's unit (the same distance in km or in m needs a different coefficient and gets penalized differently), so standardize first; choose $\alpha$ by cross-validation.

In class: R1 · Linear Regression · R2 · Gradient Descent & Regularization

TP, FP, FN and TN: true positives, false positives, false negatives and true negatives (the four cells of the confusion matrix, topic [44]).

Precision

TP/(TP+FP)

Of those I flagged positive, how many were.

Recall

TP/(TP+FN)

Of the real positives, how many I found.

F1

2PR/(P+R)

Harmonic mean of P (precision) and R (recall): high only if both are.

[39]Why is logistic regression called "regression"?C1

It fits a linear regression on the log-odds of the positive class and passes the result through a sigmoid, which turns it into a probability. It is trained with cross-entropy (log loss), not squared error. Classification only appears when a threshold is applied.

[40]How does k-NN work? How do you choose k?C2

For a new point, find the $k$ nearest training examples and let them vote (or average them, for regression). Small $k$ = high variance; large $k$ = high bias; choose it on validation. Traps: with $k = 1$ training accuracy is 100% by construction; without a scaler, the column with the largest unit decides everything; and prediction is expensive, because it measures the distance to the whole training set.

[41]What is accuracy?C3

The fraction of correct predictions. Trap: with imbalanced classes it misleads: always predicting "not fraud" is 99% right when only 1% are frauds. That is the 2 p.m. question.

[42]Precision and recall?C3

Precision: of the cases flagged as positive, how many were. Recall: of the real positives, how many were found. Raising one usually lowers the other.

[43]What is F1?C3

The harmonic mean of precision and recall. Trap: it weighs both equally and ignores the true negatives, which does not always reflect the real cost of errors (F-beta gives one side more weight).

[44]What is the confusion matrix?C3

The actual × predicted table: rows = actual class, columns = predicted class, positive first, [[TP, FN], [FP, TN]] (confusion_matrix(y, p, labels=[1, 0])). TP and TN are the hits; FP, the false alarms; FN, the missed positives. Every threshold-based metric comes from it; know how to compute them by hand.

[45]What is ROC-AUC?C4

The area under the curve of the true positive rate against the false positive rate, as the threshold varies. It is the probability that a random positive gets a higher score than a random negative. Trap: with rare events it can look excellent while precision is low.

[46]What is PR-AUC?C4

The area under the precision × recall curve (in scikit-learn, average precision). It focuses on the positive class and says more than ROC when that class is rare. The random baseline is the prevalence, not 0.5.

[47]What is class imbalance?C4

One class much rarer than the other. The model leans toward the majority and accuracy misleads. Use suitable metrics and a stratified split, and adjust the threshold or the class weights.

[48]Explain SMOTE (synthetic minority oversampling technique).C4

It creates synthetic minority examples by interpolating between an example and one of its neighbors of the same class. Training data only, inside the pipeline and the cross-validation. Trap: it can create points in the middle of the majority class; often moving the threshold or using class_weight solves the problem with less risk.

[49]class_weight, undersampling or oversampling?C4

class_weight gives the minority's errors more weight in the loss; undersampling drops majority examples (fast, loses information); oversampling duplicates or synthesizes minority ones (uses everything, risks overfitting). The last two on the training data only. Trap: all of them distort the predicted probabilities; for logistic regression, class_weight is almost the same as moving the threshold.

[50]How do you choose the threshold?C4

From the cost of each kind of error and the problem's constraints (a minimum recall, a maximum number of alerts per day). Choose it on validation, never on the test set. Trap: 0.5 is not sacred; it is just the default of predict.

In class: C1 · Logistic Regression · C2 · k-Nearest Neighbors · C3 · Classification Metrics · C4 · ROC, PR & Imbalanced Data

[51]The bias–variance trade-off?M1

Error comes from bias (a model too simple, wrong in a systematic way), variance (a model that changes a lot from one sample to another) and noise (which nothing removes). More complexity lowers bias and raises variance; the best model sits at the bottom of the U-shaped validation error curve.

[52]What is cross-validation?C5

Split the training set into $k$ parts; train on $k-1$ and validate on the remaining one, $k$ times. The mean estimates the performance and the standard deviation shows how much it varies between splits. Trap: preprocessing must restart in every fold, which is why it goes inside the pipeline.

[53]Training 0.99, validation 0.72?M1

Overfitting. First rule out leakage and a distribution difference. Then reduce variance: limit complexity, regularize, drop noisy features, collect data. See the complete example.

[54]Will more data help?M1

The learning curve answers it: training and validation error as a function of the number of training rows, with the model fixed. If there is still a gap between the curves and it is still closing at the largest size measured, more data helps (high variance). If the curves have already met at an error that is too high and gone flat, it does not (high bias): you need a more flexible model or better features. Trap: the training error rises as the training set grows (fitting 15 points is easier than fitting 800); that is expected, not a fault.

[55]Why does repeated tuning overfit the validation set?C5

Picking the best of many configurations on the same validation data partly selects the one that got lucky on it. The winner's score is optimistic. That is why the test set stays locked away.

[56]What is nested CV?C5

Two loops: the inner one picks the hyperparameters; the outer one evaluates the whole procedure on data the inner loop never saw. It gives an estimate free of the selection optimism.

[57]Grid search, random search or Bayesian optimization?E3

A grid tries every combination, but each hyperparameter at only a few values; when only a few of them matter, random search, with the same budget, explores many more values of the ones that do. Bayesian optimization (such as Optuna's TPE, the tree-structured Parzen estimator) uses the results so far to choose the next trial. Trap: the longer the search, the more optimistic its best validation score.

[58]Why can more features make things worse?C2

Irrelevant or noisy features raise variance, create spurious correlations, make distances lose meaning and make the model more expensive. They only help if they bring new information about the target.

[59]When do you prefer a simpler model?C5

When the complex model's advantage is within the variability of cross-validation (the one-standard-error rule), or when interpretability, latency, cost and maintenance weigh more than a small gain.

In class: C5 · Cross-Validation & Data Leakage · M1 · Model Selection · E3 · AutoML

[60]What does a decision tree do?M2

It splits the data recursively with "feature ≤ threshold?" questions, choosing at each step the split that most reduces impurity (Gini or entropy). Each leaf predicts the majority class or the mean. Trap: without a depth limit it memorizes the training set; and a small change in the data can produce a different tree.

[61]Explain bagging.M3

Train several models on bootstrap samples (drawn with replacement) and average them or let them vote. It reduces variance without increasing bias; it works best with unstable models, such as deep trees.

[62]What is a random forest?M3

Bagging of trees in which every split chooses among a random subset of the features. The trees become less alike, make different mistakes, and averaging cancels more of them.

[63]What is the out-of-bag (OOB) error?M3

Each bootstrap sample leaves out about 37% of the rows; the trees that never saw a row vote on it. The accuracy of those votes is a free validation, with no data set aside. Trap: only the forest has an OOB estimate; to compare it with other models, use cross-validation with the same folds for all of them. And, as with any validation, the OOB score of the winning configuration of a hyperparameter search is optimistic.

[64]Explain boosting.M4

Small trees in sequence, each one fitted to the errors of the previous ones (the residuals, which are the negative gradient of the loss with respect to the prediction), added up at the end. It reduces bias. The learning rate and the number of trees control overfitting; early stopping picks the number of trees.

[65]Random forest or gradient boosting?M4

The forest reduces variance by averaging deep, independent trees; it is hard to break and more trees never hurt. Boosting reduces bias by adding shallow trees; tuned, it is usually the best on tabular data, but bad settings can be much worse. A forest for a strong baseline with almost no tuning; boosting when the last points matter and you can validate the tuning.

[66]Tree feature importance?E2

The built-in importance (MDI, mean decrease in impurity) adds up how much each feature reduced impurity on the training set, and it rewards columns with many distinct values, even pure noise. Prefer permutation importance on held-out data: shuffle one feature and measure the drop. Trap: neither measures causation; correlated features share the credit.

[67]What is SHAP?E2

It splits one prediction into contributions, one per feature, that add up exactly to the difference between that prediction and the average prediction: they are the Shapley values of game theory. It explains one decision ("why me?") and, stacked over many rows, the whole model. Trap: it explains the model, not the world; it is not causation.

In class: M2 · Decision Trees · M3 · Random Forest · M4 · Gradient Boosting · E2 · Explainability

[68]What is the margin of an SVM? And C?C6

Among all lines that separate the classes, the SVM picks the one furthest from the nearest points; those points are the support vectors, and only they define the boundary. The soft margin accepts violations, each one charged by $C$: small $C$ = more regularization, a wide margin, many support vectors; large $C$ = an almost hard margin and a risk of overfitting.

[69]What is the kernel trick?C6

Data that no line separates can become separable in a space of transformed features; the kernel computes the similarity between points in that space without ever building it. The RBF (radial basis function, Gaussian) kernel has $\gamma$, which is coupled with $C$. Traps: scaling is mandatory; training costs between $n^2$ and $n^3$, so a kernel SVM stops being practical at hundreds of thousands of rows.

[70]Why does a neural network need a nonlinear activation?E1

Without it, a linear function of a linear function is linear: any number of layers collapses into one, which cannot even solve XOR. The nonlinearity (ReLU, the rectified linear unit $\max(0, z)$, in the hidden layers) is what lets the network draw curved boundaries.

[71]What is backpropagation? And the vanishing gradient?E1

It is the chain rule applied layer by layer, from the output back to the input, to get the gradient of the loss with respect to every weight; gradient descent then uses that gradient. Each layer multiplies the gradient by the derivative of its activation: the sigmoid's is at most 0.25, and through 10 layers that gives at most $0.25^{10} \approx 10^{-6}$, so the first layers barely learn. That is why ReLU, with slope 1 when active, became the default.

[72]How do you prevent overfitting in a neural network?E1

Weight decay (ridge's L2 penalty on every weight, scikit-learn's alpha), early stopping (hold out part of the training data and stop when its score stops improving), dropout (switch off random units at each step), fewer units and more data. And standardize the inputs.

[73]Neural network or gradient boosting on tabular data?E1

On tables of heterogeneous columns, tree ensembles usually tie or win, with less tuning: in class, on the same data and the same folds, boosting scored 0.956 ± 0.011 and the network 0.948 ± 0.005, and the network also needed a scaler, an architecture and alpha. On images, audio and text the balance tips toward networks.

[74]Data drift vs concept drift?E4

In data drift (covariate shift) the inputs change, $P(x)$, and their relationship with the target stays; in concept drift the relationship changes, $P(y \mid x)$: fraudsters change tactics, a crisis changes who repays. There is also prior shift, where the class balance changes, $P(y)$. Trap: concept drift does not show up if you only watch the inputs.

[75]How do you monitor a model when labels arrive late?E4

At two speeds. Right away: input quality checks, each feature's shift against the training data (PSI, KS test) and the distribution of the model's own scores. Later, when the labels arrive: the real metrics on each batch, compared with the validation estimate. When labels hardly ever arrive, a small audited sample is the only protection against concept drift.

[76]What is training–serving skew?E4

Any difference between how a feature is computed in training and at serving time: another unit (an amount in thousands, a duration in years), another source, other code. The model raises no error, it just predicts badly. The defense is a single pipeline on both sides and input validation against a schema.

[77]What do you save when you deploy a model?E4

The whole pipeline (preprocessing + model), not just the estimator; the library versions; the seeds; a fingerprint of the training data; the validation metrics. Trap: a pickle is a program: only load ones from a trusted source (the skops format is safer).

In class: C6 · Support Vector Machines · E1 · Neural Networks · E4 · MLOps

Quiz

Twenty-four interview-style questions, three per theme. Filter by theme to review only what you need.

Study plan

How to study, and a plan for the two weeks before an interview.

Explain

Out loud, to someone

If you cannot explain a concept to a classmate who missed the class, you have not learned it yet. Studying in pairs, one interviewing the other with the case questions, is the best practice there is.

Show it in code

Five lines from memory

Know how to write these without looking anything up: train_test_split, Pipeline, ColumnTransformer, cross_val_score, confusion_matrix, KMeans, TfidfVectorizer, permutation_importance. Redo the class labs until you can.

Space it out

A day, a week, a month

Reviewing spaced out in time sticks far better than cramming the night before. Whatever you forget between rounds is what needs the most attention.

Tick honestly

The checklist is your thermometer

Tick an item only when you can explain it without looking at anything, trap included. A checklist full of items ticked out of optimism will not help you in the interview.

Days	What to do
1–2	Fundamentals and data: key concepts and the quiz for both themes; reread classes F2, D1 and D2
3–4	Pipelines and leakage: build a leak-free pipeline from scratch without notes; explain the five leakage patterns of C5
5–6	Regression and classification: metrics by hand from a list of residuals and from a confusion matrix; ridge vs lasso; k-NN
7–8	Validation, trees and ensembles: explain bias and variance and cross-validation by drawing; forest vs boosting; permutation importance and SHAP
9	Unsupervised learning and text: k-means by hand on six points, the silhouette, PCA; TF-IDF of three sentences and the cosine between them
10	SVM, networks and production: margin and kernel, activation and backpropagation, the three kinds of drift, out loud
11–12	Case questions in pairs, timed, with the quick diagnosis; the full quiz, going back only to what you got wrong
13–14	One final round of the whole checklist, out loud

To practice in the course's format, and for the final assessment too: the workbook, with one activity per class, exercises at three levels and worked answers (in Portuguese). For a quick review: the cheat sheet, the concept map and the study cards.

Checklist

The 77 topics and the interview habits. Your progress is saved in your browser.

Back to 2 p.m.

The full answer to "is this model with 95% accuracy any good?"

1

Ask for context"What is the churn rate? If it is 5%, predicting that nobody cancels already gives 95%." — principle 1, topic [47].

2

Compare with the baseline"I would compare it with the majority class and look at the confusion matrix to see how many cancellations the model actually catches." — topics [7] and [44].

3

Change the metric"For the class that cancels, I would report recall and precision, or the PR-AUC, which is not inflated by the true negatives." — topics [41], [42] and [46].

4

Tie it to cost and propose the next step"Finally, I would choose the threshold from the cost of a wasted retention offer against that of a lost customer, on validation — and only then measure on the test set." — topic [50].

Four steps, one minute and seven of the 77 topics in a single answer. That is what the interviewer wanted to see: not what you memorized, but how you connect it.

References

To go deeper, grouped by kind.

The 2 p.m. question

What an ML interview looks like

"What is overfitting?"

"Training 0.99, validation 0.72"

"Build this pipeline"

What the interviewer is assessing

How to answer

Case questions, step by step

Quick diagnosis

"Training 0.99, validation 0.72"

What to use when

The 77 key concepts

Quiz

Study plan

Out loud, to someone

Five lines from memory

A day, a week, a month

The checklist is your thermometer

Checklist

Back to 2 p.m.

References

In this course

Books

Free online resources