Load Balancing Strategies Cheat Sheet
Covers core load balancing algorithms, Layer 4 vs Layer 7 balancing, health checks, and configuration examples using NGINX and AWS ELB.
2 PagesIntermediateJan 28, 2026
Load Balancing Algorithms
Common algorithms used to distribute traffic across backend servers.
- Round Robin- Requests distributed sequentially across all servers in order
- Weighted Round Robin- Servers with higher weight receive proportionally more requests
- Least Connections- Routes to the server with the fewest active connections
- Weighted Least Connections- Combines connection count with a server capacity weight
- IP Hash- Hashes client IP to consistently route the same client to the same server
- Least Response Time- Routes based on lowest latency plus fewest active connections
- Random- Picks a server at random, often combined with 'power of two choices'
Layer 4 vs Layer 7
Key differences between transport-layer and application-layer load balancing.
- Layer 4 (Transport)- Routes based on IP/port using TCP/UDP, no payload inspection, very fast
- Layer 7 (Application)- Routes based on HTTP headers, URL path, cookies; enables content-based routing
- AWS NLB- Layer 4 load balancer, handles millions of requests/sec with static IPs
- AWS ALB- Layer 7 load balancer, supports path/host-based routing and WebSockets
- SSL Termination- L7 LBs can decrypt TLS at the balancer, offloading CPU work from backends
NGINX Load Balancer Config
Basic upstream block for round-robin and weighted balancing.
nginx
upstream backend { least_conn; # use least connections algorithm server 10.0.0.1:8080 weight=3; server 10.0.0.2:8080 weight=1; server 10.0.0.3:8080 backup; # only used if others are down}server { listen 80; location / { proxy_pass http://backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }}
Health Check Configuration
Defining active health checks so unhealthy targets are removed automatically.
bash
# AWS CLI: configure ALB target group health checkaws elbv2 modify-target-group \ --target-group-arn arn:aws:elasticloadbalancing:...:targetgroup/my-tg \ --health-check-protocol HTTP \ --health-check-path /healthz \ --health-check-interval-seconds 15 \ --healthy-threshold-count 2 \ --unhealthy-threshold-count 3
HAProxy — Consistent Hashing for Cache Backends
Minimizes cache-key remapping when a backend is added or removed, unlike plain IP hash.
text
backend cache_pool balance hash req.hdr(X-Cache-Key) hash-type consistent server cache1 10.0.1.1:11211 check server cache2 10.0.1.2:11211 check server cache3 10.0.1.3:11211 check
Connection Draining Before Deregistration
Give in-flight requests time to complete before an instance is removed from rotation.
bash
aws elbv2 modify-target-group-attributes \ --target-group-arn arn:aws:elasticloadbalancing:...:targetgroup/my-tg \ --attributes Key=deregistration_delay.timeout_seconds,Value=30# Then deregister — ALB stops sending NEW requests immediately but# waits up to 30s for existing connections to finishaws elbv2 deregister-targets \ --target-group-arn arn:aws:elasticloadbalancing:...:targetgroup/my-tg \ --targets Id=i-0123456789abcdef0
Envoy — Outlier Detection (Circuit Breaking)
Service-mesh load balancing that ejects misbehaving upstream hosts automatically.
yaml
clusters:- name: payments-service lb_policy: LEAST_REQUEST outlier_detection: consecutive_5xx: 5 interval: 10s base_ejection_time: 30s max_ejection_percent: 50 circuit_breakers: thresholds: - priority: DEFAULT max_connections: 1000 max_pending_requests: 100 max_retries: 3
NGINX Sticky Sessions (Cookie-Based)
Pins a client to the same backend for session-affine apps that don't share session state externally.
nginx
upstream backend { ip_hash; # simplest sticky option, hashes client IP}# OR, cookie-based affinity via the njs / sticky module:upstream backend_cookie { server 10.0.0.1:8080; server 10.0.0.2:8080; sticky cookie srv_id expires=1h domain=.example.com path=/;}
Advanced Load Balancing Patterns
Concepts for scaling load balancing beyond a single tier or region.
- GSLB (Global Server Load Balancing)- DNS-based routing across regions using latency, geo-proximity, or health (e.g. Route 53 latency routing)
- Active-Passive Failover- Standby region/backend only receives traffic when health checks fail on the primary
- Power of Two Choices- Sample two random backends, pick the less-loaded one; near-optimal balance with O(1) cost, no global state
- PROXY Protocol- Preserves original client IP/port when traffic passes through an L4 LB before an L7 proxy
- TLS Passthrough vs Termination- Passthrough forwards encrypted bytes untouched (LB can't inspect); termination decrypts at the LB and re-encrypts (or not) to backends
- Weighted Traffic Shifting- Canary/blue-green rollout by gradually shifting a percentage of traffic to a new target group
- Session Draining vs Draining Timeout- Distinguish app-level graceful shutdown handlers from the LB's own deregistration delay — both must be tuned together
Pro Tip
Set your health check interval and timeout tighter than your deployment rollout speed, otherwise the LB will keep sending traffic to instances mid-restart during a rolling deploy.
Was this cheat sheet helpful?
Explore Topics
#LoadBalancingStrategies#LoadBalancingStrategiesCheatSheet#CloudComputing#Intermediate#LoadBalancingAlgorithms#Layer4VsLayer7#NGINX#Load#Algorithms#CheatSheet#SkillVeris
