Scikit-Learn for Beginners: Machine Learning in Python
SkillVeris Team
Data Science Team
Scikit-learn's API is elegantly consistent: every model has fit(X, y), predict(X), and score(X, y).
In this guide, you'll learn:
- Train on training data, evaluate on test data, and use cross-validation for reliable performance estimates.
- Switching algorithms is just changing the class name — the rest of the code stays the same.
- Always split train/test before any preprocessing, and fit scalers on the training data only to avoid data leakage.
- For imbalanced classes use precision, recall, F1, and ROC-AUC instead of accuracy; for regression use RMSE, MAE, and R2.
1What Is Scikit-Learn?
Scikit-learn (sklearn) is the standard Python library for classical machine learning: linear regression, logistic regression, decision trees, random forests, support vector machines, k-means clustering, and more.
It provides a clean, consistent API that makes switching between algorithms easy, plus excellent tools for model evaluation, preprocessing, and pipeline construction.
Common Imports
A typical set of imports for a sklearn project.
import sklearn
print(sklearn.__version__) # 1.4.x or later
from sklearn.linear_model import LogisticRegression, LinearRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import accuracy_score, classification_report
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder2The Consistent API
Every scikit-learn model follows the same three-method API: fit to train, predict on new data, and score to evaluate. Switching algorithms is just changing the class name — the rest of the code stays the same. This uniformity is sklearn's greatest strength.
fit, predict, score
The same three methods work across every estimator.
# Create model
model = RandomForestClassifier(n_estimators=100, random_state=42)
# Train on labelled data
model.fit(X_train, y_train)
# Predict on new data
predictions = model.predict(X_test)
# Evaluate
score = model.score(X_test, y_test) # accuracy for classifiers, R2 for regressors3Loading and Preparing Data
Scikit-learn ships with built-in datasets for learning, and works seamlessly with pandas DataFrames loaded from CSV. The library expects X with shape (n_samples, n_features) and y with shape (n_samples,).
Loading Built-in and CSV Data
Separate the feature matrix X from the target y.
from sklearn.datasets import load_iris, load_diabetes
import pandas as pd
# Built-in datasets for learning
iris = load_iris(as_frame=True)
df = iris.frame
X = iris.data # feature matrix: (150, 4)
y = iris.target # labels: 0, 1, 2
# From a CSV
df = pd.read_csv("data.csv")
X = df.drop("target", axis=1) # all columns except target
y = df["target"] # target column
# sklearn expects: X shape (n_samples, n_features), y shape (n_samples,)
print(X.shape, y.shape) # e.g. (150, 4) (150,)4Train/Test Split
Never evaluate a model on data it was trained on — that gives optimistically biased results. Always hold out a test set and use it only for final evaluation. The stratify argument preserves class proportions across both splits.
⚠️Watch Out
Data leakage happens when information from the test set influences training. Common causes: fitting preprocessing (StandardScaler, PCA) on all data instead of training data only; including future data in features; target leakage (a feature only known after the target is known). Leakage produces models that look great in evaluation but fail in production.
Splitting the Data
An 80/20 split with a fixed random_state for reproducibility.
from sklearn.model_selection import train_test_split
# Split 80% train, 20% test
X_train, X_test, y_train, y_test = train_test_split(
X, y,
test_size=0.2, # 20% for testing
random_state=42, # reproducible split
stratify=y # preserve class proportions in both splits
)
print(f"Train: {X_train.shape[0]} samples")
print(f"Test: {X_test.shape[0]} samples")5Classification: Predicting Categories
Classification predicts discrete categories. The example below trains a Random Forest on the breast cancer dataset and measures accuracy, which typically lands around 0.96.
A Random Forest Classifier
Fit, predict, and score in a few lines.
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
data = load_breast_cancer()
X, y = data.data, data.target
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y)
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
preds = model.predict(X_test)
acc = accuracy_score(y_test, preds)
print(f"Accuracy: {acc:.3f}") # typically ~0.966Model Evaluation for Classification
Accuracy alone can mislead. For imbalanced datasets (90% class 0, 10% class 1), a model that always predicts "class 0" gets 90% accuracy. Use precision, recall, F1-score, and ROC-AUC instead, alongside a confusion matrix.
Classification Metrics
A full report, confusion matrix, and ROC-AUC.
from sklearn.metrics import (classification_report, confusion_matrix,
ConfusionMatrixDisplay, roc_auc_score)
# Full classification report
print(classification_report(y_test, preds, target_names=data.target_names))
# Confusion matrix
cm = confusion_matrix(y_test, preds)
disp = ConfusionMatrixDisplay(cm, display_labels=data.target_names)
disp.plot()
# ROC-AUC (for binary classification with probability scores)
proba = model.predict_proba(X_test)[:, 1] # probability of positive class
auc = roc_auc_score(y_test, proba)
print(f"ROC-AUC: {auc:.3f}")7Regression: Predicting Numbers
Regression predicts continuous numbers. Ridge adds regularisation to linear regression, while Gradient Boosting is usually the best performer on tabular data.
Ridge and Gradient Boosting
Two regressors on the diabetes dataset.
from sklearn.datasets import load_diabetes
from sklearn.linear_model import Ridge
from sklearn.ensemble import GradientBoostingRegressor
data = load_diabetes()
X, y = data.data, data.target
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42)
# Linear regression with regularisation
ridge = Ridge(alpha=1.0)
ridge.fit(X_train, y_train)
# Gradient boosting (usually best for tabular data)
gb = GradientBoostingRegressor(n_estimators=200, learning_rate=0.1, random_state=42)
gb.fit(X_train, y_train)
preds = gb.predict(X_test)8Model Evaluation for Regression
Regression metrics quantify error in the target's units. RMSE and MAE measure average error, while R2 ranges from 1.0 (perfect) down through 0 (baseline) to negative (worse than predicting the mean).
RMSE, MAE, and R2
Three complementary regression metrics.
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import numpy as np
preds = gb.predict(X_test)
rmse = np.sqrt(mean_squared_error(y_test, preds))
mae = mean_absolute_error(y_test, preds)
r2 = r2_score(y_test, preds)
print(f"RMSE: {rmse:.2f}") # same units as target
print(f"MAE: {mae:.2f}") # average absolute error
print(f"R2: {r2:.3f}") # 1.0 = perfect, 0 = baseline, negative = worse than mean9Feature Engineering and Preprocessing
Scaling is required for SVM, KNN, and linear models, and categorical features must be encoded. Crucially, fit the scaler on training data and only transform the test data. ColumnTransformer applies different transforms to different columns.
Scaling and Encoding
Fit on train, transform on test, and combine with ColumnTransformer.
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
# Scale numeric features (required for SVM, KNN, linear models)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # fit + transform on train
X_test_scaled = scaler.transform(X_test) # transform only on test
# One-hot encode categorical features
encoder = OneHotEncoder(sparse_output=False, handle_unknown="ignore")
# ColumnTransformer: apply different transforms to different columns
numeric_cols = ["age", "salary"]
categorical_cols = ["dept", "city"]
preprocessor = ColumnTransformer([
("num", StandardScaler(), numeric_cols),
("cat", OneHotEncoder(handle_unknown="ignore"), categorical_cols),
])10Cross-Validation
Cross-validation trains and evaluates the model multiple times on different splits, giving a reliable estimate of real-world performance. The mean and standard deviation tell you both how good the model is and how stable it is across data splits.
Stratified K-Fold Cross-Validation
Five folds give a mean and a spread.
from sklearn.model_selection import cross_val_score, StratifiedKFold
model = RandomForestClassifier(n_estimators=100, random_state=42)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(model, X, y, cv=cv, scoring="accuracy")
print(f"CV scores: {scores}")
print(f"Mean: {scores.mean():.3f} +/- {scores.std():.3f}")11Pipelines
Pipelines are the correct way to handle preprocessing: by including the scaler inside the pipeline, cross-validation fits the scaler on training folds only, preventing leakage. Always use pipelines in production code.
Building and Saving a Pipeline
fit, predict, and score work exactly the same on a pipeline.
from sklearn.pipeline import Pipeline
# Chain preprocessing + model into one object
pipe = Pipeline([
("scaler", StandardScaler()),
("model", RandomForestClassifier(n_estimators=100, random_state=42)),
])
# fit/predict/score work exactly the same
pipe.fit(X_train, y_train)
preds = pipe.predict(X_test)
print(f"Pipeline accuracy: {pipe.score(X_test, y_test):.3f}")
# Cross-validate the whole pipeline (no leakage)
scores = cross_val_score(pipe, X, y, cv=5)
print(f"CV mean: {scores.mean():.3f}")
# Save and load
import joblib
joblib.dump(pipe, "model.joblib")
pipe_loaded = joblib.load("model.joblib")12Key Takeaways
The sklearn workflow is consistent and predictable once these habits are in place.
- Every sklearn model: fit(X_train, y_train) -> predict(X_test) -> score(X_test, y_test).
- Always split train/test before any preprocessing; fit scalers on training data only.
- For imbalanced classes, use F1/AUC not accuracy; for regression, use RMSE/MAE/R2.
- Cross-validation gives more reliable estimates than a single train/test split.
- Use Pipelines to chain preprocessing and modelling into one leak-proof, reusable object.
13What to Learn Next
Go deeper with machine learning using these next steps.
- Machine Learning for Beginners — the conceptual foundation behind these algorithms.
- NumPy for Data Science — the array operations sklearn operates on.
- Pandas for Beginners — load and prepare real-world data for sklearn.
14Frequently Asked Questions
What is the difference between fit() and fit_transform()? fit() learns parameters from data (e.g. the mean and standard deviation for StandardScaler). transform() applies the learned transformation. fit_transform() does both in one step. Use fit_transform() on training data, then transform() only on test data — never refit on test data.
When should I use Random Forest vs Gradient Boosting vs Linear models? Linear models (Ridge, Logistic Regression) are fast, interpretable, and a good baseline that works well with many features. Random Forest is robust, handles missing values, has low hyperparameter sensitivity, and is a good default for most tabular problems. Gradient Boosting (XGBoost, LightGBM) is usually the best performer on tabular data but slower to train and more sensitive to hyperparameters. Start with a linear baseline, then try Random Forest, then Gradient Boosting.
What is overfitting and how do I detect it? Overfitting occurs when a model memorises training data but fails to generalise. Signs: training accuracy much higher than test accuracy (e.g. 99% train, 72% test). Solutions: more data, regularisation (Ridge, Lasso, max_depth on trees), cross-validation to detect it, and simpler models.
Does scikit-learn support deep learning? No. Scikit-learn covers classical machine learning. For deep learning (neural networks, computer vision, NLP), use PyTorch or TensorFlow/Keras. The two ecosystems complement each other: sklearn for classical ML, PyTorch for deep learning, and sklearn's preprocessing and evaluation tools can be used alongside PyTorch.
Related Reading
Get The Print Version
Download a PDF of this article for offline reading.
About the Publisher
SkillVeris Team
Data Science Team
Our data team shares real-world analytics, ML, and SQL insights grounded in industry practice.
View all postsRelated Posts
Never miss an update
Get the latest tutorials and guides delivered to your inbox.
No spam. Unsubscribe anytime.