Skip to content

ML Pipeline Simulator

Loading the simulator… (it needs JavaScript)

Reference

Your own data: files and links

In the Data block, press Your data…. There are two ways in.

Load a file from your computer

Click Choose a file… or drag the file onto the dashed box.

  • Formats: CSV, TSV or TXT with any separator (it is detected; a file separated by ;, as Excel exports in many locales, is read with a decimal comma), also gzipped (.gz), Excel (.xlsx), Parquet or JSON.
  • The first row must hold the column names.
  • Size: up to 100 MB; above about 20 MB it gets slow. For large datasets, use Rows used (a sample) while you experiment.
  • Privacy: the file is read inside your browser and is not sent to any server. It stays in memory while the page is open; if you reload the page, choose it again.

Load by link

Paste the address in the link field and press Load link. A browser can only read a file from another site if that site allows it (the rule is called CORS), so not every link works.

Source Works? What to paste
GitHub ✓ The link of the file page (…/blob/…) or the raw link; the simulator converts it
Gist ✓ The raw link
Google Drive ✓ Share as Anyone with the link and paste the file link (…/file/d/…/view)
Google Sheets ✓ Share as Anyone with the link and paste the sheet link; the open tab is exported as CSV
Hugging Face ✓ The …/resolve/… link (or …/blob/…, which is converted)
Kaggle ✗ Needs a login: download the file and choose it from your computer
Dropbox, OneDrive, UCI and many university sites ✗ They do not let another page read the file: download it and choose it from your computer

A link that returns a web page instead of the file (the Drive preview page, for instance) is rejected with a message explaining what to do. The exported code reads the data from the same link.

Once loaded, the simulator lists each column with its type (number or category), how many values are missing and how many distinct values it has. Pick the target column, the task (classification or regression) and, in a binary classification, the positive class. Untick the columns that should stay out (identifiers, free text) and click the type of a numeric column to treat it as a category (like Pclass in the Titanic data).

How the pipeline is built

The simulator always builds the same scikit-learn skeleton, the one from the preprocessing and pipelines lessons:

prep = ColumnTransformer([
    ("num", Pipeline([("impute", ...), ("transform", ...), ("scale", ...)]), num_cols),
    ("cat", Pipeline([("impute", SimpleImputer(strategy="most_frequent")), ("encode", ...)]), cat_cols),
])
pipe = Pipeline([("prep", prep), ("reduce", ...), ("model", ...)])
  • Per column (in the Preparation node) lets a column leave its lane, with its own imputation, transform and scaling (or another encoding, if it is categorical). Columns with the same choices become one branch of the ColumnTransformer, such as ("num_log1p", Pipeline([...]), num_log1p_cols) in the exported code.
  • The Explore tab shows the training rows (X_train) before any model: the target, each column (histogram by class, skewness, outliers, missing values), the association between columns and suggestions with buttons that change the pipeline. Each pair of columns is measured the way its types allow: Pearson's r for number × number, the correlation ratio η for category × number and Cramér's V (bias-corrected) for category × category; clicking a cell draws the pair as a scatter, one box per category or the make-up of each category. The test rows stay out here too; whatever you decide by looking at the data has not peeked at them.
  • The train/test split comes first (train_test_split, stratified in a classification). Everything is fitted on the training rows only: the imputer's mean, the scaling, PCA, feature selection and the model.
  • Cross-validation runs on the training rows; the test set is held out and evaluated once. The metrics table shows the three columns side by side: training far above validation is overfitting.
  • Permutation importance shuffles one original column at a time in the test rows and measures how much the main metric drops.
  • The Fit the preprocessing on ALL rows option is there for you to watch leakage happen: with it on, the preprocessing sees the test rows before the evaluation, and the numbers become optimistic.
  • Compare models puts several models through the same preparation and the same folds (cross_val_score for each) and shows every fold's score; Use puts the chosen one in the pipeline.
  • Grid search runs a GridSearchCV over one or two hyperparameters (up to 100 combinations): with one, it draws the validation curve; with two, a heat map. The best combination is scored on the test set once, and Apply to the pipeline copies the values into the model.
  • The Code button gives, in Python, what the open tab shows: the last run, the comparison or the grid search.
  • Download the prepared data (in the Preparation block) and Download the reduced data (in the Reduction block) save a .csv with every row as the model receives it: the steps are fitted on the training rows, the split column marks train and test and the last column is the target.
  • The 2D map tab draws the training rows in two dimensions with t-SNE (scikit-learn), UMAP (umap-js, because umap-learn needs numba, which Pyodide lacks) or PCA's first two components, after Preparation or after Reduction too, colored by the target. It is only for looking: nothing there enters the pipeline. Above 50 columns, t-SNE and UMAP first get 50 principal components.
  • The browser runs scikit-learn 1.8. Colab may run another version, so the last decimals can differ.
Scenarios
Scenario What it teaches Lesson
k-NN without scaling Distances dominated by the largest-scale column k-NN
The accuracy trap 98% accuracy that misses the rare cases; class weight and threshold ROC and imbalance
The tree that memorizes Training vs validation, depth and pruning Decision trees
Titanic from scratch Missing values, categories, identifiers that blow up one-hot Preprocessing
Leakage: learning from noise Feature selection outside cross-validation Cross-validation
A polynomial too flexible Degree, regularization and the validation curve Model selection
Pricing houses Imputation, categories and a non-linear model Linear regression
64 pixels, 20 numbers PCA before k-NN Dimensionality reduction
Bending the boundary A linear boundary vs kernels, neighbors and polynomial features SVM

Your pipeline is saved in this browser for each scenario; ↺ Reset brings the original back.

What the simulator is (and is not)

It is a lab for training the reasoning of a machine learning project: which step solves which problem, how to read training vs validation vs test, and how to compare alternatives fairly. It covers the tabular path of the lessons (with scikit-learn, XGBoost and LightGBM), not text, large images or deep learning. The computing is your own computer's: large datasets and heavy models (an SVM on tens of thousands of rows, forests of hundreds of trees under cross-validation) can take minutes; the â–  Stop button restarts Python.