Markov Chains Cheat Sheet

Explains states, transition matrices, and the Markov property, and shows how to simulate chains and compute stationary distributions in Python.

2 PagesIntermediateMar 18, 2026

Core Concepts

Foundational vocabulary for Markov chains.

Markov property- The future state depends only on the current state, not the full history ("memorylessness")
State space- The set of all possible states the system can occupy
Transition matrix P- P[i][j] = probability of moving from state i to state j; each row sums to 1
Stationary distribution- A distribution pi such that pi*P = pi; describes the chain's long-run behavior if it exists and is unique
Ergodic chain- Irreducible and aperiodic chain that converges to a unique stationary distribution regardless of start state
Absorbing state- A state that, once entered, cannot be left (P[i][i] = 1)

Simulating a Markov Chain

Sample a sequence of states from a transition matrix.

python

import numpy as npstates = ['Sunny', 'Rainy', 'Cloudy']P = np.array([    [0.7, 0.2, 0.1],    [0.3, 0.4, 0.3],    [0.2, 0.3, 0.5],])def simulate(start_idx, n_steps, P):    current = start_idx    path = [current]    for _ in range(n_steps):        current = np.random.choice(len(P), p=P[current])        path.append(current)    return [states[i] for i in path]print(simulate(0, 10, P))

Stationary Distribution

Solve pi*P = pi via the eigenvector for eigenvalue 1.

python

import numpy as npeigvals, eigvecs = np.linalg.eig(P.T)stationary = eigvecs[:, np.isclose(eigvals, 1)]stationary = (stationary / stationary.sum()).real.flatten()print(dict(zip(states, stationary)))# Alternative: repeatedly multiply P by itself for large powers and inspect any row

Applications

Where Markov chains show up in data science.

PageRank- Random surfer model treats web pages as states and links as transitions
Hidden Markov Models- Extend chains with unobserved states inferred from observed emissions (speech recognition, POS tagging)
MCMC sampling- Markov Chain Monte Carlo builds a chain whose stationary distribution is the target posterior
Customer journey modeling- Model transitions between marketing touchpoints or app screens

Pro Tip

Before trusting a stationary distribution, verify the chain is irreducible (every state reachable from every other) and aperiodic -- otherwise pi*P = pi may not be unique or may never be reached from your starting state.

Was this cheat sheet helpful?

Survival Analysis Cheat Sheet

Monte Carlo Simulation Cheat Sheet

Explore Topics

#MarkovChains#MarkovChainsCheatSheet#DataScience#Intermediate#CoreConcepts#SimulatingAMarkovChain#StationaryDistribution#Applications#MachineLearning#CheatSheet#SkillVeris

Sri Hayavadhana Info-Tech

Professional Web Designing Services