Machine Learning Quick Reference
This reference consolidates the vocabulary, formulas, and decision rules that recur across a machine learning curriculum into one scannable page. It is meant to be revisited throughout a course — while other topics build depth on a single concept, this one is intentionally broad and shallow, trading detail for speed of lookup. Use it to jog your memory on a definition mid-project, or as a final review before an assessment or interview.
Cricket analogy: This page is like a cricket almanac's quick-reference stat sheet - not deep analysis of any one player, just fast lookup of averages and strike rates to jog your memory mid-match discussion or before a big series.
Core Terminology
A feature is an input variable used to make a prediction; a label (or target) is the value being predicted. A model is trained (fit) on a training set, tuned using a validation set, and given a final, unbiased performance check on a test set it has never influenced in any way. A hyperparameter (e.g., the number of trees in a random forest, or the learning rate) is set before training and is not learned from data, unlike a parameter (e.g., a regression coefficient), which is learned during fit().
Cricket analogy: A feature is a stat like strike rate predicting the label, runs scored; a batsman trains in the nets, gets tuned in warm-up matches (validation), and is judged in the actual tournament (test) he's never faced; practice sessions are a hyperparameter, while his learned timing is like a parameter.
Algorithm Cheat-Sheet
For regression tasks: linear regression is the fast, interpretable baseline; ridge/lasso add regularization to control overfitting; random forests and gradient boosting handle non-linearity and interactions with less feature engineering. For classification: logistic regression is the interpretable baseline; k-nearest neighbors is simple but scales poorly with large datasets; decision trees are interpretable but overfit easily alone; random forests and gradient boosting (e.g., XGBoost) are strong general-purpose choices; support vector machines work well on smaller, high-dimensional datasets. For clustering: k-means assumes roughly spherical, similarly sized clusters and requires choosing k in advance; DBSCAN finds arbitrarily shaped clusters and automatically labels outliers as noise; hierarchical clustering produces a dendrogram useful when the number of clusters is unknown.
Cricket analogy: Predicting a batter's final score is like linear regression's job, a fast baseline, while predicting a five-wicket haul is a classification call for logistic regression or a random forest; grouping bowlers into styles without labels is clustering, like k-means grouping similar spin bowlers.
Metric Cheat-Sheet
For regression: RMSE (root mean squared error) penalizes large errors heavily and is in the same units as the target; MAE (mean absolute error) is more robust to outliers; R-squared expresses the proportion of variance explained. For classification: accuracy is only meaningful with balanced classes; precision = TP / (TP + FP) answers 'of predicted positives, how many were correct'; recall = TP / (TP + FN) answers 'of actual positives, how many did we catch'; F1 is the harmonic mean of precision and recall; ROC-AUC measures ranking quality across all thresholds and is less sensitive to class imbalance than accuracy, though PR-AUC is often preferred for severe imbalance.
Cricket analogy: RMSE punishes a wildly wrong score prediction, like missing a Rohit Sharma century by 60 runs, heavily and stays in run units; MAE forgives that one outlier innings more; R-squared shows variance explained. For man-of-the-match, accuracy misleads with a clear favorite, so precision, recall, F1, and ROC-AUC matter more.
Formulas at a Glance
Gradient descent update: w := w - learning_rate * gradient(loss, w). Sigmoid: 1 / (1 + e^-x). Mean squared error: (1/n) * sum((y_true - y_pred)^2). L1 penalty (lasso): lambda * sum(|w_i|). L2 penalty (ridge): lambda * sum(w_i^2). Bias-variance decomposition: expected test error = bias^2 + variance + irreducible error.
Cricket analogy: Gradient descent nudges weights opposite the gradient by a learning rate, like a coach tweaking technique each session rather than an overhaul. Sigmoid squashes a score into a win probability, MSE averages squared errors, L1/L2 stop overreacting to one flashy innings, and bias-variance explains consistent versus inconsistent errors.
from sklearn.linear_model import LinearRegression, LogisticRegression, Ridge, Lasso
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
from sklearn.cluster import KMeans, DBSCAN
from sklearn.metrics import (
mean_squared_error, r2_score,
accuracy_score, precision_score, recall_score, f1_score, roc_auc_score,
)
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
# Quick lookup: typical pattern for any supervised model
# X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# model = RandomForestClassifier(n_estimators=200, max_depth=8, random_state=42)
# model.fit(X_train, y_train)
# preds = model.predict(X_test)
# print('accuracy:', accuracy_score(y_test, preds))
# print('f1:', f1_score(y_test, preds))Rule of thumb for picking a first model: start with the simplest interpretable baseline (linear/logistic regression), get an evaluation pipeline correct first, then move to more complex models (random forest, gradient boosting) only if the simple baseline underperforms your requirements.
This page is a memory aid, not a substitute for understanding why each formula or rule holds. Relying on it without having worked through the corresponding full topic risks applying a metric or algorithm correctly by name but incorrectly in context.
- Features are inputs, labels are targets; parameters are learned during training, hyperparameters are set beforehand.
- Linear/logistic regression are strong interpretable baselines; random forests and gradient boosting are strong general-purpose defaults.
- K-means needs k chosen in advance and assumes spherical clusters; DBSCAN handles arbitrary shapes and flags noise automatically.
- Precision, recall, and F1 matter more than accuracy under class imbalance; ROC-AUC and PR-AUC evaluate ranking quality.
- Core formulas: gradient descent update, sigmoid, MSE, L1/L2 penalties, and the bias-variance decomposition.
- Always validate a chosen model/metric against the actual problem context — a cheat sheet is a memory jog, not a decision-maker.
Practice what you learned
1. What is the key difference between a model parameter and a hyperparameter?
2. According to the algorithm cheat-sheet, why might DBSCAN be preferred over k-means for a clustering task?
3. Which metric formula correctly represents recall?
4. What does the bias-variance decomposition state about expected test error?
5. Per the quick-reference rule of thumb, what is a sensible first step when starting a new supervised learning project?
Was this page helpful?
You May Also Like
Intro to scikit-learn and the ML Toolchain
Scikit-learn provides a consistent fit/predict interface for building, evaluating, and chaining machine learning models, and anchors the wider Python ML toolchain.
Confusion Matrix and Classification Metrics
A structured breakdown of correct and incorrect predictions by class, forming the foundation for accuracy, precision, recall, and other classification metrics.
Bias-Variance Tradeoff
A foundational concept explaining how model error decomposes into bias, variance, and irreducible noise, and how balancing them guides model complexity choices.
Machine Learning Interview Questions
A curated set of frequently asked ML interview questions with concise, technically precise answers spanning fundamentals, algorithms, and evaluation.