Survival Analysis Cheat Sheet
Covers censoring, the survival and hazard functions, Kaplan-Meier estimation, and Cox proportional hazards regression using the lifelines library.
Core Concepts
Vocabulary specific to time-to-event data.
- Censoring- When the event (e.g., death, churn, failure) hasn't occurred by the end of observation; right-censoring is most common
- Survival function S(t)- Probability that the event has not yet occurred by time t
- Hazard function h(t)- Instantaneous risk of the event occurring at time t, given survival up to t
- Kaplan-Meier estimator- Non-parametric estimate of the survival function from censored data
- Proportional hazards assumption- Cox model assumption that covariates' effect on the hazard ratio is constant over time
- Log-rank test- Hypothesis test comparing survival curves between two or more groups
Kaplan-Meier Estimator
Estimate and plot the survival curve for a cohort.
from lifelines import KaplanMeierFitterkmf = KaplanMeierFitter()kmf.fit(durations=df['time'], event_observed=df['event'], label='All customers')kmf.plot_survival_function()print(kmf.median_survival_time_)
Cox Proportional Hazards Model
Model how covariates affect the hazard rate.
from lifelines import CoxPHFittercph = CoxPHFitter()cph.fit(df, duration_col='time', event_col='event')cph.print_summary() # coefficients, hazard ratios, p-valuescph.plot() # forest plot of hazard ratios# hazard ratio > 1 => higher risk; < 1 => protective effecthr = cph.hazard_ratios_
When to Reach for Survival Analysis
Signals that ordinary regression is the wrong tool.
- Time-to-churn modeling- Predicting when a subscriber will cancel rather than just whether they will
- Equipment failure / reliability- Estimating time until a machine part fails
- Clinical trials- Comparing time-to-event (e.g., relapse) between treatment and control groups
- Right-censored data present- Use survival methods instead of dropping or imputing censored rows, which biases estimates
Parametric Weibull AFT Model
Fit an Accelerated Failure Time model when a parametric survival distribution is a reasonable assumption.
from lifelines import WeibullAFTFitteraft = WeibullAFTFitter()aft.fit(df, duration_col='time', event_col='event')aft.print_summary()# Coefficients are on the time scale: exp(coef) > 1 speeds up (shortens) survival time,# exp(coef) < 1 decelerates (lengthens) survival time -- opposite interpretation to Cox hazard ratiosmedian_survival = aft.predict_median(df)survival_curve = aft.predict_survival_function(df.iloc[[0]])
Time-Varying Covariates
Model covariates that change during follow-up (e.g., a customer's usage tier) with a long-format Cox model.
from lifelines import CoxTimeVaryingFitter# long format: one row per (id, interval), with start/stop columns# id start stop event usage_tier# 1 0 30 0 'free'# 1 30 90 1 'paid'ctv = CoxTimeVaryingFitter()ctv.fit( long_df, id_col='id', event_col='event', start_col='start', stop_col='stop')ctv.print_summary()
Checking the Proportional Hazards Assumption
Use Schoenfeld residuals to test whether a fitted Cox model's covariate effects are truly constant over time.
from lifelines import CoxPHFittercph = CoxPHFitter()cph.fit(df, duration_col='time', event_col='event')# statistical test + plots per covariate; p < 0.05 suggests the assumption is violatedcph.check_assumptions(df, p_value_threshold=0.05, show_plots=True)# fix for a violating covariate: stratify on it instead of including it linearlycph_strat = CoxPHFitter()cph_strat.fit(df, duration_col='time', event_col='event', strata=['region'])
Competing Risks with Cumulative Incidence
Model multiple mutually-exclusive event types (e.g., churn vs. upgrade) where one event precludes the other.
from lifelines import AalenJohansenFitterajf = AalenJohansenFitter()# event_col: 0 = censored, 1 = churn, 2 = upgrade (competing event)ajf.fit(durations=df['time'], event_observed=df['event_type'], event_of_interest=1)ajf.plot(label='Cumulative incidence of churn')# Naively treating competing events as censoring with a standard KM estimator# overestimates the probability of the event of interest.
Evaluating Survival Models
Metrics for comparing survival models beyond the log-rank test.
- Concordance index (C-index)- Probability that, for a random pair, the model ranks the subject with the shorter observed survival time as higher risk; analogous to AUC for time-to-event data
- Time-dependent AUC- C-index generalization evaluated at a specific horizon t, useful when discrimination changes over follow-up time
- Integrated Brier Score- Time-averaged squared error between predicted and observed survival probability, rewarding both discrimination and calibration
- Restricted Mean Survival Time (RMST)- Area under the survival curve up to a fixed horizon; a robust summary that doesn't require the proportional hazards assumption
- Random survival forests- Ensemble of survival trees (scikit-survival's RandomSurvivalForest) that captures nonlinear/interaction effects the Cox model's linear hazard misses
Never drop censored observations to run ordinary linear regression -- that discards the very information (that the event hadn't happened yet) survival models are built to use correctly.