Machine Learning Foundations in R
R's machine learning ecosystem spans three layers: base R functions like lm() and glm() for classical statistical models, algorithm-specific packages like randomForest, xgboost, and e1071 for specific model families, and unifying frameworks like caret and the newer tidymodels that provide one consistent interface — train() or fit() — across dozens of underlying algorithms. This layered design means you can start with a simple glm() baseline and swap in a gradient-boosted tree later without rewriting your evaluation code, because the unifying framework handles data splitting, resampling, and metric calculation identically regardless of which algorithm sits underneath.
Cricket analogy: Switching from a glm() baseline to xgboost under the same caret framework is like a T20 franchise swapping an all-rounder for a specialist finisher like Andre Russell without changing the batting order — the team structure (evaluation code) stays intact.
Building Predictive Models
Logistic regression via glm(outcome ~ ., data = df, family = "binomial") is the standard first model for binary classification in R because its coefficients are directly interpretable as log-odds, and exp(coef(fit)) converts them into odds ratios that stakeholders can reason about. Predictions come from predict(fit, newdata, type = "response"), which returns probabilities that must be thresholded (commonly at 0.5, but tuned via ROC analysis for imbalanced data) to produce a class label.
Cricket analogy: Interpreting a glm() coefficient as an odds ratio is like a commentator saying a team's win probability shifts by a fixed multiple for every extra wicket lost in the powerplay — a clean, explainable number, unlike a black-box model.
randomForest(outcome ~ ., data = df, ntree = 500) builds an ensemble of decision trees, each trained on a bootstrap sample with a random subset of predictors considered at each split, and averages their votes (for classification) or predictions (for regression) to reduce the overfitting any single tree would suffer. Calling importance(model) or varImpPlot(model) ranks predictors by how much they reduce impurity or increase out-of-bag error when permuted, giving a built-in feature-importance report that a single decision tree or linear model can't easily replicate.
Cricket analogy: A random forest averaging 500 trees each trained on a bootstrap sample is like a T20 team's final batting order being decided by aggregating 500 simulated matches rather than trusting one selector's gut call on one game.
Model Evaluation and Cross-Validation
The caret package standardizes model training through trainControl(method = "cv", number = 10) for 10-fold cross-validation and train(outcome ~ ., data = df, method = "rf", trControl = ctrl), which fits the specified algorithm on each fold and averages the held-out performance to give a realistic estimate of how the model will generalize to new data. After training, confusionMatrix(predict(model, test), test$outcome) reports accuracy, sensitivity, specificity, and Cohen's kappa, giving a much fuller picture of classifier performance than accuracy alone, especially when classes are imbalanced.
Cricket analogy: 10-fold cross-validation is like judging a batsman's true ability by rotating them through 10 different pitch conditions — Chepauk, the WACA, Eden Gardens — rather than trusting a single flat-track hundred as proof of skill.
The tidymodels Framework
The tidymodels framework decomposes the workflow into composable pieces: a recipe() object defines preprocessing steps like step_normalize() and step_dummy(), a parsnip model specification like logistic_reg() %>% set_engine("glm") defines the algorithm, and a workflow() bundles both together so that preprocessing and modeling are always applied consistently across training and test data. This separation prevents a common source of bugs — accidentally normalizing training and test data using different statistics — and makes it trivial to swap set_engine("glm") for set_engine("ranger") to try a random forest without touching the preprocessing recipe.
Cricket analogy: Bundling a recipe() and a model spec into one workflow() is like a franchise locking in both its fielding drills and its batting order into one match-day plan, so nothing gets applied inconsistently between the warm-up and the actual game.
library(caret)
library(randomForest)
# Logistic regression baseline
glm_fit <- glm(purchased ~ age + income + visits, data = train_df, family = "binomial")
probs <- predict(glm_fit, newdata = test_df, type = "response")
preds <- ifelse(probs > 0.5, "yes", "no")
# Random forest with feature importance
rf_fit <- randomForest(purchased ~ age + income + visits, data = train_df, ntree = 500, importance = TRUE)
varImpPlot(rf_fit)
# 10-fold cross-validated training via caret
ctrl <- trainControl(method = "cv", number = 10, classProbs = TRUE, summaryFunction = twoClassSummary)
cv_model <- train(purchased ~ age + income + visits, data = train_df,
method = "rf", trControl = ctrl, metric = "ROC")
confusionMatrix(predict(cv_model, test_df), test_df$purchased)When classes are imbalanced (e.g., 2% fraud rate), accuracy is misleading — a model that predicts 'not fraud' every time is 98% accurate but useless. Use caret's summaryFunction = twoClassSummary to optimize for ROC/AUC instead, and consider resampling techniques like sampling = "smote" or sampling = "down" inside trainControl() to rebalance the training folds.
Tuning and Comparing Models
Hyperparameter tuning in caret is done by passing a tuneGrid data frame — for a random forest, expand.grid(mtry = c(2, 4, 6)) — to train(), which fits the model at every combination within each cross-validation fold and selects the value that maximizes the chosen metric. For comparing several fully-tuned models, resamples(list(rf = rf_model, glm = glm_model, xgb = xgb_model)) collects their cross-validated performance distributions so bwplot() or dotplot() can visualize which model is genuinely better rather than better by chance on one split.
Cricket analogy: Grid-searching mtry values in caret is like a coach trialing three different fielding formations across several practice matches to see which minimizes runs conceded, rather than guessing which formation is best from one game.
A model that scores near-perfect accuracy on cross-validation can still fail in production if there's data leakage — for example, if a step_normalize() recipe was fit on the full dataset before splitting into train/test, information from the test set leaks into training. Always split first, then fit preprocessing steps only on the training fold, which is exactly what a tidymodels workflow() enforces automatically when used with fit_resamples().
- R's ML ecosystem layers base functions, algorithm-specific packages, and unifying frameworks (
caret,tidymodels) that share one training interface. glm(family = "binomial")gives an interpretable logistic regression baseline with coefficients readable as odds ratios viaexp(coef(fit)).randomForest()ensembles hundreds of bootstrap-trained trees and reports feature importance viaimportance()/varImpPlot().caret::trainControl(method = "cv")standardizes k-fold cross-validation andconfusionMatrix()reports accuracy, sensitivity, specificity, and kappa.tidymodelsseparates preprocessing (recipe), algorithm (parsnip), and their combination (workflow) to prevent train/test leakage.- Hyperparameters are tuned via
tuneGridincaretortune_grid()intidymodels, selecting the value that maximizes cross-validated performance. - Always fit preprocessing steps on the training fold only — fitting on the full dataset before splitting causes data leakage and inflated performance estimates.
Practice what you learned
1. Which family argument makes `glm()` fit a logistic regression model for binary outcomes?
2. In `randomForest()`, what does each individual tree in the ensemble train on?
3. Why is accuracy alone a misleading metric for a highly imbalanced classification problem, such as 2% fraud rate?
4. In `tidymodels`, what is the purpose of bundling a `recipe()` and a `parsnip` model spec into a `workflow()`?
5. What does passing a `tuneGrid` to `caret::train()` accomplish?
Was this page helpful?
You May Also Like
Statistical Analysis in R
Learn how to compute descriptive statistics, run hypothesis tests, and fit regression models using R's base stats package.
Testing R Code with testthat
Learn how to write, organize, and run automated unit tests for R functions and packages using the testthat framework.
Writing R Packages
Learn how to structure, document, and build a distributable R package using usethis, roxygen2, and devtools.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics