1. EDA

Project: Exploratory Data Analysis (EDA)

First deliverable of the classification or regression project. The team explores the dataset it will use until the end of the semester, finds out what is hard about it and ends with a preprocessing pipeline ready for modeling. No model is trained in this deliverable.

0. Proposal

The team picks the task and submits through the form up to 3 datasets, in order of preference, each with: URL, size (rows × columns), target, number of numerical and categorical features, a 2–3 sentence motivation and the first risk it sees. The instructor approves one — an EDA on a dataset that was not approved is not graded.

1. Initial inspection

• A. Data dictionary — where the dataset comes from, what each row is and a table with the meaning, type and unit of each feature.
• B. Quality — missing values per column (count and %), duplicates, impossible or inconsistent values and columns to drop: identifiers, constants and those that leak the target.
• C. Target — classification: frequency of each class and imbalance; regression: statistics, histogram and skewness.
• D. Train and test — split with a fixed seed, stratified (or temporal) when appropriate. From here on, every preprocessing statistic is computed on the training set only.

Feature types · Quality · Leakage · Imbalance · Splitting

2. Univariate analysis

• A. Numerical — descriptive table for all of them (mean, median, deviation, quartiles) and figures interpreting the shape of each distribution: skewness, modes, outliers.
• B. Categorical — frequencies and cardinality for all of them and bar charts, pointing out rare and high-cardinality categories.

Distributions

3. Bivariate and multivariate analysis

• A. Numerical × numerical — correlation matrix (Pearson or Spearman, justified) and scatter plots, pointing out redundant pairs.
• B. Categorical × target — class proportions per category (classification) or target distribution per category (regression).
• C. Numerical × categorical — grouped boxplots: do the groups differ in location, in spread, or in both?

Every figure ends with one explicit conclusion.

4. Preprocessing

Every choice points to the finding that motivates it: "we standardized because the features range from 0–1 to 0–10⁵ (Table 3)", not just "we standardized".

• A. Strategies — missing values, outliers, categorical encoding (including a category that is new in the test set) and scaling, keeping in mind the destination is a neural network.
• B. Dimensionality reduction — three 2D projections of the scaled training features, colored by the target:
  • PCA: cumulative explained variance and a reading of the loadings;
  • t-SNE and UMAP: at least 2 values of perplexity / n_neighbors each; on large datasets, use a sample;
  • compare: what do the nonlinear projections reveal that PCA does not? In t-SNE and UMAP, cluster sizes and the distances between clusters have no direct reading.
• C. Pipeline — Pipeline + ColumnTransformer fitted on train only, in an importable file under code/, reporting that no NaN is left, the final shape and the feature names.

Preprocessing · Distributions (PCA, t-SNE)

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline

preprocess = ColumnTransformer([
    ("num", Pipeline([...]), numeric_features),
    ("cat", Pipeline([...]), categorical_features),
])

X_train_t = preprocess.fit_transform(X_train)   # fit on train only
X_test_t = preprocess.transform(X_test)         # test is only transformed

5. Synthesis

A short section, written for whoever will build the model: the main findings, each with the figure or table that supports it, and the risks for modeling with a handling plan. Close with the table below filled in — it is the index of the numbers the report has already computed.

Submission

• Report at docs/projects/eda/index.md, in one team repository (given in the proposal), with headings in the order of this page — see Submission Format;
• The deadline is not extended · plagiarism zeroes the deliverable · AI is allowed, but every member explains each number and each decision — oral exams may happen and the Project Evaluation caps the grade.

---
project: eda
task: classification          # or regression
dataset: https://...
team:
  - First Last
ai_use: "none"
---

Rubric

Each item is scored in full, by half or zero: in full when it is complete and justified; half when the justification or the number supporting the analysis is missing; zero when missing or wrong. An approved proposal is a prerequisite — without it, the EDA is not graded.