Skip to content

References

Textbooks

  • Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. Springer, 2009. Free at hastie.su.domains/ElemStatLearn.
  • James, G.; Witten, D.; Hastie, T.; Tibshirani, R. An Introduction to Statistical Learning with Applications in Python. Springer, 2023. Free at statlearning.com.
  • Bishop, C. M. Pattern Recognition and Machine Learning. Springer, 2006.
  • Murphy, K. P. Probabilistic Machine Learning: An Introduction. MIT Press, 2022. Free at probml.github.io/pml-book.
  • Géron, A. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. 3rd ed. O'Reilly, 2022.
  • Mitchell, T. M. Machine Learning. McGraw-Hill, 1997 — the classic definition of a learning program.

Papers & historical sources

Regression

  • Legendre, A.-M. Nouvelles méthodes pour la détermination des orbites des comètes. Courcier, 1805 — first publication of least squares, in the appendix Sur la Méthode des moindres quarrés. Full text (archive.org)
  • Gauss, C. F. Theoria motus corporum coelestium in sectionibus conicis solem ambientium. Perthes & Besser, 1809 — where Gauss states he had used least squares since 1795, opening the priority dispute with Legendre. Full text (archive.org)
  • Galton, F. "Regression towards Mediocrity in Hereditary Stature." Journal of the Anthropological Institute 15 (1886): 246–263 — the paper that gave regression its name. PDF (galton.org) · DOI
  • Hanley, J. A. "'Transmuting' Women into Men: Galton's Family Data on Human Stature." The American Statistician 58, no. 3 (2004): 237–243 — the modern reanalysis that recovers Galton's moments and explains his mid-parent construction. DOI
  • Anscombe, F. J. "Graphs in Statistical Analysis." The American Statistician 27, no. 1 (1973): 17–21 — the quartet. DOI
  • Cook, R. D. "Detection of Influential Observation in Linear Regression." Technometrics 19, no. 1 (1977): 15–18 — Cook's distance. DOI
  • Hoerl, A. E.; Kennard, R. W. "Ridge Regression: Biased Estimation for Nonorthogonal Problems." Technometrics 12, no. 1 (1970): 55–67. DOI

Other historical sources

  • Turing, A. M. "Computing Machinery and Intelligence." Mind (1950).
  • Rosenblatt, F. "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain." Psychological Review (1958).
  • Cortes, C.; Vapnik, V. "Support-Vector Networks." Machine Learning (1995).
  • Breiman, L. "Random Forests." Machine Learning (2001).
  • Friedman, J. H. "Greedy Function Approximation: A Gradient Boosting Machine." Annals of Statistics (2001).
  • Chen, T.; Guestrin, C. "XGBoost: A Scalable Tree Boosting System." KDD (2016).
  • Lundberg, S.; Lee, S.-I. "A Unified Approach to Interpreting Model Predictions." NeurIPS (2017) — SHAP.
  • Grootendorst, M. "BERTopic: Neural topic modeling with a class-based TF-IDF procedure." arXiv:2203.05794 (2022).

Software & documentation

  • scikit-learn — the reference library for classical ML in Python.
  • pandas and NumPy — data manipulation.
  • XGBoost / LightGBM — gradient boosting.
  • BERTopic — modern topic modeling.
  • SHAP — model explainability.
  • Optuna — hyperparameter optimization.