Practice — House Price Prediction Pipeline — Machine Learning with Scikit-learn

What You'll Build

In this exercise you will build a complete end-to-end regression pipeline for house price prediction, applying every technique from this module: OLS and regularised regression with cross-validated hyperparameter selection, polynomial feature expansion for non-linear relationships, comprehensive evaluation with MAE, RMSE, and R2, residual diagnostics to verify assumptions, and a final report that interprets coefficients and identifies systematic failure patterns.

The scenario uses a synthetic dataset inspired by real estate markets, with features covering area, location score, age, proximity to amenities, and several engineered features. The exercise is structured in four steps: foundation (data understanding and splitting), core logic (training and comparing multiple regression approaches), integration (diagnostics and threshold analysis), and testing and reporting (final model evaluation and interpretation). The cricket analogy throughout is predicting match venue scores — the same modelling workflow applies.

Analogy🏏Cricket

🏏 Think of it like cricket: Predicting house prices is structurally identical to predicting match scores: the features (area, location, age) play the role of match conditions (pitch, weather, opposition), and the goal is to build a pipeline that makes the most accurate predictions while being interpretable enough for stakeholders (buyers, sellers, regulators) to trust. Just as a match-score predictor must pass diagnostic checks to ensure it models pitch and weather effects correctly, a house-price model must pass residual diagnostics before being used for financial decisions.

Prerequisites

Python 3.10 or later with NumPy, pandas, and scikit-learn installed.
Command of OLS linear regression and its assumptions from Lesson 07.
Understanding of Ridge, Lasso, and ElasticNet from Lesson 08.
Familiarity with polynomial regression from Lesson 09.
Mastery of MAE, RMSE, and R2 from Lesson 10 and residual diagnostics from Lesson 11.

Setup and Dataset

Generate the synthetic house-price dataset with realistic properties: right-skewed price distribution, a non-linear area-price relationship (larger houses cost disproportionately more per square metre), and multicollinear features. Apply an immediate log transform to the target to address right-skew, which is standard practice for price modelling.

Analogy🏏Cricket

🏏 Think of it like cricket: setting up the house-price dataset is like surveying a lopsided pool of player valuations before you start modelling. Prices are right-skewed — a handful of superstar mansions tower over a mass of ordinary homes, just as a few marquee players' fees dwarf the rest. The area-price link is non-linear, larger houses costing disproportionately more per square metre, like elite all-rounders commanding a premium that balloons rather than adds up. And features are multicollinear, several overlapping stats telling the same story. Applying an immediate log transform to the skewed target is like switching valuations to a compressed scale so the giants no longer distort the whole picture, exactly as analysts log-transform lopsided fees before comparing them. Just as a scout first tames a skewed valuation pool into a fair, comparable scale before ranking anyone, you log the target upfront. The payoff: a well-conditioned dataset where later modelling isn't dominated by a few extreme outliers.

python

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(42)
n = 800

# House price dataset with realistic structure
area_sqm   = rng.uniform(40, 400, n)
age_years  = rng.uniform(0, 50, n)
location   = rng.uniform(1, 10, n)         # neighbourhood score
amenity_km = rng.uniform(0.5, 15, n)       # distance to nearest amenity
floors     = rng.integers(1, 6, n).astype(float)
renovated  = rng.choice([0,1], n, p=[0.7, 0.3])

# True price: non-linear in area, exponential in location, additive in others
log_price = (
    0.6 * np.log(area_sqm)         # concave in area (diminishing returns)
    + 0.25 * location
    - 0.008 * age_years
    - 0.03 * amenity_km
    + 0.07 * floors
    + 0.08 * renovated
    + 9.0                           # intercept (log scale)
    + rng.normal(0, 0.15, n)       # noise
)
price_lakhs = np.exp(log_price)

df = pd.DataFrame({
    "area_sqm": area_sqm, "age_years": age_years, "location": location,
    "amenity_km": amenity_km, "floors": floors, "renovated": renovated,
    "price_lakhs": price_lakhs,
})

print(f"Dataset: {df.shape}")
print(f"Price range: {price_lakhs.min():.0f} - {price_lakhs.max():.0f} lakhs")
print(f"Price median: {np.median(price_lakhs):.0f} lakhs")
print(f"Price skewness: {pd.Series(price_lakhs).skew():.2f} (right-skewed -> log-transform)")

# LOG TRANSFORM the target (standard for right-skewed prices)
df["log_price"] = np.log(df["price_lakhs"])
print(f"Log-price skewness: {df['log_price'].skew():.2f} (much more symmetric)")

Step 1 — Foundation: Split and Baseline

Establish a stratified-by-quantile train-test split and compute the naive baseline — predicting the training-set mean log-price for every observation. This baseline defines the floor that any model must beat, and computing it before any modelling makes all subsequent comparisons grounded in a concrete reference point.

Analogy🏏Cricket

🏏 Think of it like cricket: establishing your split and baseline is like sealing an honest final trial and first pinning down the dumbest possible benchmark to beat. A stratified-by-quantile split ensures your locked-away test set holds a fair mix of cheap, mid, and premium homes — like guaranteeing your final selection trial contains budget players, solid pros, and superstars in proportion, so it fairly represents the whole market. The naive baseline — always predicting the training mean log-price — is the equivalent of a lazy pundit who forecasts 'about the average valuation' for every player regardless of talent. Just as a selector fixes that floor first so every real model must visibly clear it to justify its complexity, you compute the baseline before any modelling. The payoff: a concrete reference point that grounds every later comparison, ensuring you can prove your fancy pipeline actually beats simply guessing the average rather than merely sounding impressive.

python

from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np

features = ["area_sqm", "age_years", "location", "amenity_km", "floors", "renovated"]
X = df[features].values
y = df["log_price"].values   # predict log-price, transform back for reporting

# Stratify by log-price quantile to preserve price distribution
price_quantile = pd.qcut(y, q=4, labels=False)
X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.20, stratify=price_quantile, random_state=42)

# NAIVE BASELINE: predict training mean for every observation
y_baseline = np.full(len(y_te), y_tr.mean())
baseline_mae  = mean_absolute_error(y_te, y_baseline)
baseline_rmse = np.sqrt(mean_squared_error(y_te, y_baseline))
baseline_r2   = r2_score(y_te, y_baseline)

print(f"Train: {len(X_tr)}, Test: {len(X_te)}")
print(f"\nNaive baseline (predict mean log-price):")
print(f"  MAE={baseline_mae:.4f}, RMSE={baseline_rmse:.4f}, R2={baseline_r2:.4f}")
print(f"  In original scale: predictions = exp({y_tr.mean():.3f}) = "
      f"{np.exp(y_tr.mean()):.0f} lakhs for every house")
print("Any model must beat this baseline to demonstrate learning.")

Step 2 — Core Logic: Train and Compare Regression Models

Train and compare four regression approaches using five-fold cross-validation on the training set: plain OLS (the reference), Ridge with CV-tuned alpha, Lasso with CV-tuned alpha, and a polynomial Ridge (degree two) to capture the non-linear area-price relationship. All pipelines include standardisation to enable valid regularisation.

Analogy🏏Cricket

🏏 Think of it like cricket: comparing four regression approaches under five-fold cross-validation is like trialling four coaching philosophies across rotating practice rounds before committing. Plain OLS is the reference baseline coach; Ridge with CV-tuned alpha is a coach who gently reins in every attribute so no single stat dominates; Lasso with CV-tuned alpha is a ruthless selector who zeroes out the useless features entirely; and polynomial Ridge of degree two is a coach who deliberately teaches the curved, non-linear technique needed for the accelerating area-price relationship. All of them standardise their inputs first so the regularisation judges every feature fairly. Just as a head selector puts rival coaching plans through the same rotating trials and keeps the one that consistently wins across rounds, you cross-validate all four on the training set alone. The payoff: an evidence-based choice of model, proven over multiple folds rather than crowned by a single lucky split.

python

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, PolynomialFeatures
from sklearn.linear_model import LinearRegression, RidgeCV, LassoCV
from sklearn.linear_model import Ridge
from sklearn.model_selection import cross_val_score, StratifiedKFold
import numpy as np

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

models = {
    "OLS":           Pipeline([("sc", StandardScaler()), ("m", LinearRegression())]),
    "Ridge (CV)":    Pipeline([("sc", StandardScaler()), ("m", RidgeCV(alphas=np.logspace(-3,3,30), cv=5))]),
    "Lasso (CV)":    Pipeline([("sc", StandardScaler()), ("m", LassoCV(cv=5, random_state=42))]),
    "Poly+Ridge":    Pipeline([("sc", StandardScaler()),
                               ("poly", PolynomialFeatures(degree=2, include_bias=False)),
                               ("m", RidgeCV(alphas=np.logspace(-2,4,30), cv=5))]),
}

print(f"{'Model':15} | {'CV MAE':>8} | {'CV RMSE':>9} | {'CV R2':>7}")
print("-" * 50)
best_r2, best_name, best_pipe = -999, None, None
for name, pipe in models.items():
    cv_mae  = -cross_val_score(pipe, X_tr, y_tr, cv=cv, scoring="neg_mean_absolute_error").mean()
    cv_rmse = np.sqrt(-cross_val_score(pipe, X_tr, y_tr, cv=cv, scoring="neg_mean_squared_error").mean())
    cv_r2   = cross_val_score(pipe, X_tr, y_tr, cv=cv, scoring="r2").mean()
    print(f"{name:15} | {cv_mae:>8.4f} | {cv_rmse:>9.4f} | {cv_r2:>7.4f}")
    if cv_r2 > best_r2:
        best_r2, best_name, best_pipe = cv_r2, name, pipe
print(f"\nBest model: {best_name} (CV R2={best_r2:.4f})")

Step 3 — Integration: Diagnostics and Coefficient Interpretation

Fit the best model on the full training set, run residual diagnostics to validate assumptions, and interpret the standardised coefficients. For a log-price target, coefficients on standardised features represent the log-price change per standard deviation of each feature — a one-standard-deviation increase in area increases log-price by the area coefficient, which corresponds approximately to a percentage increase in price equal to (exp(coef) - 1) * 100.

Analogy🏏Cricket

🏏 Think of it like cricket: this step fits your chosen model on the full training pool, then interrogates it like a coach validating and explaining his final verdict. Running residual diagnostics is the video review confirming the model's assumptions genuinely hold before you trust it — no hidden systematic flaw lurking. Interpreting standardised coefficients is like stating precisely how much each attribute moves a player's valuation: on a log-price target, a coefficient tells you the log-price change per one-standard-deviation rise in that feature, so a one-SD bump in area lifts log-price by the area coefficient — which translates roughly into a percentage change in actual price. Just as a selector who can say 'one extra level of fitness adds about 8% to a player's value' commands more trust than a black-box hunch, standardised coefficients make each factor's effect legible. The payoff: a validated model whose every driver you can explain in plain, percentage terms.

python

from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import scipy.stats as stats
import numpy as np

best_pipe.fit(X_tr, y_tr)
y_pred_tr = best_pipe.predict(X_tr)
residuals = y_tr - y_pred_tr

# RESIDUAL DIAGNOSTICS
print("=== Residual Diagnostics ===")
print(f"Mean residual: {residuals.mean():.5f} (should be ~0)")
print(f"Residual std:  {residuals.std():.4f}")

# Heteroscedasticity check
low_mask  = y_pred_tr < np.median(y_pred_tr)
print(f"Std (low fitted):  {residuals[low_mask].std():.4f}")
print(f"Std (high fitted): {residuals[~low_mask].std():.4f}")

# Normality check
_, p_shapiro = stats.shapiro(residuals[:100])
print(f"Shapiro-Wilk p: {p_shapiro:.4f} ({'normal' if p_shapiro>0.05 else 'non-normal'})")

# COEFFICIENT INTERPRETATION (for OLS / linear model steps)
# Get the final model's coefficients (after polynomial expansion if applicable)
final_model = best_pipe[-1]
if hasattr(final_model, "coef_"):
    n_features_out = len(final_model.coef_)
    print(f"\nModel has {n_features_out} coefficients (after any polynomial expansion).")
    if n_features_out == len(features):
        for feat, coef in zip(features, final_model.coef_):
            pct_effect = (np.exp(coef) - 1) * 100
            print(f"  {feat:12}: coef={coef:+.4f} -> {pct_effect:+.1f}% price per 1-std change")

Step 4 — Testing and Final Report

Evaluate the best model on the held-out test set, report all metrics in both the log scale and the original price scale, and perform a final segmented error analysis to check whether the model performs equally across price ranges. Convert log-price predictions back to lakhs for reporting: if the model predicts log_price = 4.5, the predicted price is exp(4.5) = 90.02 lakhs.

Analogy🏏Cricket

🏏 Think of it like cricket: the final report opens the sealed trial exactly once and translates the verdict back into language everyone understands. You evaluate the chosen model on the held-out test set and report metrics in both the log scale you trained on and the original price scale stakeholders actually feel — converting a predicted log-price of 4.5 back to exp(4.5) = 90.02 lakhs, just as an analyst converts an abstract rating back into a real transfer fee people can act on. The segmented error analysis, checking performance across cheap, mid, and premium homes, is like verifying your valuation model is equally sharp for budget players and superstars rather than nailing the average while botching the extremes. Just as a selector confirms his method works across every tier of talent, not merely on typical players, you check errors across price ranges. The payoff: a single honest final measurement, reported in real rupees and proven fair across the whole market.

python

import numpy as np
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

# FINAL TEST SET EVALUATION — one look only
y_te_pred = best_pipe.predict(X_te)

# Metrics in log scale
mae_log  = mean_absolute_error(y_te, y_te_pred)
rmse_log = np.sqrt(mean_squared_error(y_te, y_te_pred))
r2_log   = r2_score(y_te, y_te_pred)

# Convert back to original scale
price_true = np.exp(y_te)
price_pred = np.exp(y_te_pred)
mae_orig  = mean_absolute_error(price_true, price_pred)
rmse_orig = np.sqrt(mean_squared_error(price_true, price_pred))

print("=" * 60)
print("FINAL TEST SET RESULTS")
print("=" * 60)
print(f"Log-scale:     MAE={mae_log:.4f}  RMSE={rmse_log:.4f}  R2={r2_log:.4f}")
print(f"Original scale: MAE={mae_orig:.1f} lakhs  RMSE={rmse_orig:.1f} lakhs")

# Segmented error analysis by price quartile
quartiles = np.percentile(price_true, [25, 50, 75])
labels = ["Budget (<25th pct)", "Mid-range", "Upper-mid", "Premium (>75th pct)"]
bins = [-np.inf] + list(quartiles) + [np.inf]
for i, label in enumerate(labels):
    mask = (price_true >= bins[i]) & (price_true < bins[i+1])
    if mask.sum() > 0:
        seg_mae = mean_absolute_error(price_true[mask], price_pred[mask])
        print(f"  {label:25}: MAE={seg_mae:.1f} lakhs  (n={mask.sum()})")

print("\nSegmented analysis reveals if model fails for specific price ranges.")

Warning: When using a log-transformed target, always remember to exponentiate predictions before reporting them to stakeholders or comparing them in original-scale business terms. Reporting log-price predictions to a home buyer or a banker is meaningless and potentially misleading. Also, the RMSE in log scale is not directly comparable to RMSE in the original price scale — always convert back for any business communication.

Extension Challenge: Add two engineered features to the pipeline: area-squared (to capture the non-linear diminishing-returns relationship between area and price that the log transform only partially addresses) and an interaction between area and location score (larger houses in better locations command a price premium that neither variable alone captures). Retrain all models with these added features and compare CV R2 to determine whether the new features improve predictive performance. Report which model benefits most from the engineered features.

Log-transforming right-skewed price targets is the standard approach, converting multiplicative effects to additive and making errors percentage-based.
Polynomial features capture the non-linear area-price relationship; Ridge regularisation handles the resulting correlated expanded features.
The naive baseline (predict training mean) is the essential reference point — all model metrics must be compared against it.
Residual diagnostics validate assumptions and are required before reporting model results in any consequential application.
Coefficients on standardised features with a log target represent log-price changes per standard deviation, convertible to percentage price effects.
Segmented error analysis by price quartile reveals whether the model systematically fails for specific price ranges.
Always exponentiate log-price predictions before reporting to stakeholders in original-scale (lakhs or crores) terms.