Monte Carlo Simulation Cheat Sheet
Covers the core idea of Monte Carlo estimation, random sampling, and variance reduction, with Python examples for integration and probability estimation.
2 PagesIntermediateMar 15, 2026
Core Concepts
The building blocks of Monte Carlo methods.
- Law of large numbers- The sample average converges to the expected value as the number of samples grows
- Monte Carlo estimator- Approximates an expectation or integral by averaging over random samples
- Variance reduction- Techniques (antithetic variates, control variates, importance sampling, stratified sampling) that reduce estimator variance for a given sample size
- Convergence rate- Standard Monte Carlo error shrinks as O(1/sqrt(n)) regardless of dimensionality
- Random seed- Fixing the seed makes simulations reproducible
Estimating Pi
Classic example: estimate pi by sampling points in a unit square.
python
import numpy as npnp.random.seed(42)n = 1_000_000x, y = np.random.uniform(-1, 1, n), np.random.uniform(-1, 1, n)inside_circle = (x**2 + y**2) <= 1pi_estimate = 4 * inside_circle.sum() / nprint(f"Estimated pi: {pi_estimate:.5f}")
Monte Carlo Integration
Approximate a definite integral by averaging function values at random points.
python
import numpy as npdef f(x): return np.sin(x) * np.exp(-x / 5)a, b, n = 0, 10, 100_000samples = np.random.uniform(a, b, n)integral_estimate = (b - a) * np.mean(f(samples))std_error = (b - a) * np.std(f(samples)) / np.sqrt(n)print(f"Integral estimate: {integral_estimate:.4f} +/- {1.96*std_error:.4f}")
Variance Reduction Techniques
Get a tighter estimate without more samples.
- Antithetic variates- Pair each random sample u with 1-u to cancel out some sampling error
- Control variates- Adjust the estimate using a correlated variable whose expectation is known exactly
- Importance sampling- Sample more often from regions that contribute most to the estimate, then reweight
- Stratified sampling- Divide the domain into strata and sample each proportionally, reducing variance from clumping
Importance Sampling
Reweight samples from an easy-to-sample proposal q(x) to estimate an expectation under a hard target p(x).
python
import numpy as npfrom scipy import stats# Estimate E_p[h(x)] where p is standard normal, using a heavier-tailed proposal qn = 200_000q = stats.t(df=3) # proposal: Student-t (easy to sample, covers the tails well)p = stats.norm() # target distributionsamples = q.rvs(n)weights = p.pdf(samples) / q.pdf(samples) # importance weightsh = samples ** 2 # e.g. estimate E_p[X^2] = 1estimate = np.mean(weights * h)ese = np.sum(weights) ** 2 / np.sum(weights ** 2) # effective sample sizeprint(f"Estimate: {estimate:.4f}, Effective sample size: {ese:.0f} / {n}")
Control Variates
Reduce variance by subtracting a correlated quantity whose expectation is known in closed form.
python
import numpy as npnp.random.seed(0)n = 100_000u = np.random.uniform(0, 1, n)# Target: integral of exp(x) over [0,1]. Control variate: X itself, E[U] = 0.5 (known)f = np.exp(u)control = ucov = np.cov(f, control)[0, 1]var_control = np.var(control)c_star = -cov / var_control # optimal coefficientf_cv = f + c_star * (control - 0.5) # control-variate-adjusted estimatorprint(f"Naive MC: {f.mean():.5f} (var={f.var():.6f})")print(f"Control variate: {f_cv.mean():.5f} (var={f_cv.var():.6f})")
Metropolis-Hastings MCMC
Build a Markov chain whose stationary distribution is a target density known only up to a normalizing constant.
python
import numpy as npdef target_unnorm(x): # e.g. unnormalized bimodal density return np.exp(-0.5 * (x - 3) ** 2) + np.exp(-0.5 * (x + 3) ** 2)def metropolis_hastings(n_samples, step_size=1.0, x0=0.0): x = x0 samples = np.empty(n_samples) for i in range(n_samples): proposal = x + np.random.normal(0, step_size) accept_ratio = target_unnorm(proposal) / target_unnorm(x) if np.random.uniform() < min(1.0, accept_ratio): x = proposal # accept samples[i] = x # reject keeps previous sample return samplessamples = metropolis_hastings(50_000, step_size=2.0)print(f"Posterior mean estimate: {samples[5000:].mean():.3f}") # burn-in first 5000
Advanced Sampling Techniques
Beyond plain Monte Carlo, for when convergence or dimensionality is a bottleneck.
- Quasi-Monte Carlo- Uses low-discrepancy sequences (Sobol, Halton) instead of pseudo-random numbers for O(1/n) convergence in low-to-moderate dimensions
- Rejection sampling- Sample from an envelope distribution and accept with probability p(x)/(M*q(x)); simple but wastes samples if M is loose
- Sequential Monte Carlo (particle filters)- Propagate a weighted population of samples through time, resampling to avoid weight degeneracy -- used for state-space model inference
- Common random numbers- Reuse the same random seed across two simulated scenarios being compared, so the variance of the difference is reduced even if individual variances aren't
- Rao-Blackwellization- Replace a Monte Carlo estimate of a sub-quantity with its exact conditional expectation whenever a closed form exists, which provably reduces variance
- Bootstrap resampling- Resample the observed data with replacement to approximate the sampling distribution of a statistic without parametric assumptions
Quasi-Monte Carlo with Sobol Sequences
Replace pseudo-random uniforms with a low-discrepancy sequence for faster convergence on smooth integrands.
python
import numpy as npfrom scipy.stats import qmcdef f(x): return np.prod(np.cos(2 * np.pi * x), axis=1)d = 4 # dimensionalitysampler = qmc.Sobol(d=d, scramble=True, seed=42)m = 16 # 2^m pointsu = sampler.random_base2(m=m)qmc_estimate = f(u).mean()# Compare to plain Monte Carlo with the same sample countrng = np.random.default_rng(42)u_mc = rng.uniform(size=(2 ** m, d))mc_estimate = f(u_mc).mean()print(f"QMC estimate: {qmc_estimate:.6f}, MC estimate: {mc_estimate:.6f}")
Pro Tip
Always report the standard error (or a confidence interval) alongside a Monte Carlo estimate -- the point estimate alone hides how many samples you'd need to trust the third decimal place.
Was this cheat sheet helpful?
Explore Topics
#MonteCarloSimulation#MonteCarloSimulationCheatSheet#DataScience#Intermediate#CoreConcepts#EstimatingPi#MonteCarloIntegration#VarianceReductionTechniques#MachineLearning#CheatSheet#SkillVeris
