Why WebSockets Don't Scale Like HTTP
A stateless HTTP request can be answered by any server in a pool because nothing about the request depends on prior interactions; the load balancer just picks the next healthy node. A WebSocket connection is the opposite: once the TCP handshake and the HTTP Upgrade complete, that specific socket lives on that specific process for as long as the client stays connected, often minutes or hours. If you naively add more WebSocket servers behind a round-robin balancer, each new HTTP request (including reconnects) might land on a different instance, but a message a client sends over its already-open socket only ever reaches the one process holding that socket. Scaling therefore isn't just about adding capacity for new connections; it's about making sure any server can deliver a message to any connected client regardless of which process physically holds their socket.
Cricket analogy: It's like a stadium where each spectator is handed a walkie-talkie tuned to one specific usher's channel at the gate, similar to how Virat Kohli's fan section only hears announcements from their assigned steward, not from ushers stationed elsewhere in Eden Gardens.
Sticky Sessions and Their Limits
The simplest scaling fix is sticky sessions: the load balancer inspects a cookie or hashes the client IP so that every request from the same client, including the initial Upgrade request, always routes to the same backend instance. This works for basic horizontal scaling of connection count, but it has real limits. If that instance crashes or is taken down during a rolling deploy, every socket pinned to it drops simultaneously and reconnecting clients get redistributed, which can cause a stampede on the remaining nodes. Sticky sessions also make autoscaling awkward, because a newly spun-up instance receives zero traffic until existing connections churn, and they don't solve the harder problem of one client needing to receive a message that originated from an action taken by a different client connected to a different server.
Cricket analogy: It's like the BCCI assigning each district's ticket queries to one specific call center desk based on pincode; the system works day-to-day, but if that desk shuts for the evening, every caller from that district is suddenly stranded and floods whichever desk picks up next.
Sharing State Across Instances
The real solution is decoupling delivery from the socket's physical location. Every WebSocket server subscribes to a shared message broker, commonly Redis Pub/Sub, Kafka, or NATS, and publishes any inbound message it receives to that broker rather than trying to deliver it directly. Every other server subscribes to relevant channels and, when a message arrives, checks whether it currently holds a socket for the intended recipient and pushes it down that socket if so. This turns the cluster into a mesh where any instance can originate a message and any instance can deliver it, completely decoupling 'which server received the message' from 'which server holds the recipient's connection'. Frameworks like Socket.IO formalize this with an 'adapter' abstraction (its Redis adapter is the most common example) so application code doesn't need to know about the broker at all.
Cricket analogy: It's like the BCCI's central scoreboard feed: every stadium's electronic board doesn't generate its own score, it subscribes to one shared feed, so a six hit in Mumbai instantly updates the display in Chennai without the two grounds talking directly.
// Node.js WebSocket server using Socket.IO's Redis adapter
const { Server } = require('socket.io');
const { createAdapter } = require('@socket.io/redis-adapter');
const { createClient } = require('redis');
const pubClient = createClient({ url: 'redis://redis:6379' });
const subClient = pubClient.duplicate();
await Promise.all([pubClient.connect(), subClient.connect()]);
const io = new Server(httpServer, {
adapter: createAdapter(pubClient, subClient),
});
// This now broadcasts to sockets held by ANY instance in the cluster,
// not just the one that received the emit call.
io.to(`room:${roomId}`).emit('message', payload);Rule of thumb for capacity planning: a single Node.js process handling lightweight JSON WebSocket traffic can typically sustain 10,000-50,000 concurrent idle connections before file-descriptor limits, event-loop latency, or memory (each socket has real per-connection buffer overhead) become the bottleneck. Load-test with realistic message rates, not just connection counts, before setting per-instance capacity targets.
Scaling Patterns
In practice, teams combine a few patterns rather than picking one. A gateway/broker split separates 'connection holding' servers (thin, stateless-ish processes that just manage sockets) from 'business logic' servers that process messages, communicating over the same Redis or Kafka backbone; this lets you scale connection capacity independently of compute-heavy processing. Presence tracking, knowing which users are online and on which node, is usually implemented with a shared key-value store (Redis with TTL-based keys is common) rather than in-memory maps, so any instance can answer 'is this user online' correctly. For routing efficiency at very large scale, some systems use consistent hashing on user ID to decide which shard of the broker's channels a server subscribes to, avoiding every instance subscribing to every channel, which becomes wasteful past a few hundred thousand concurrent users.
Cricket analogy: It's like the IPL splitting broadcast duties: Star Sports' outside-broadcast vans handle the raw camera feeds (connection holding) while a separate production hub in Mumbai does graphics and analysis (business logic), scaling each independently based on match demand.
Rolling deployments are a common source of production incidents with stateful WebSocket servers. Simply killing an old pod sends an abrupt close frame to every connection it holds, causing a synchronized reconnect storm. Use connection draining: stop routing new connections to the instance, send clients a custom 'please reconnect' message so they can migrate gracefully over a window (e.g., 30-60 seconds), and only terminate the process after active connections drop to zero or a hard timeout is reached.
- WebSocket connections are stateful and pinned to one process, unlike stateless HTTP requests.
- Sticky sessions (cookie or IP-hash based) are the simplest scaling step but cause reconnect stampedes on instance failure or deploy.
- A shared broker (Redis Pub/Sub, Kafka, NATS) decouples message origin from socket location, letting any server deliver to any connected client.
- Socket.IO's Redis adapter is a common off-the-shelf implementation of the publish/subscribe fan-out pattern.
- Presence data should live in a shared store like Redis with TTLs, not in per-instance memory.
- Splitting connection-holding gateways from business-logic servers lets you scale each independently.
- Rolling deploys need connection draining to avoid synchronized reconnect storms.
Practice what you learned
1. Why can't a plain round-robin load balancer scale WebSocket servers the way it scales stateless HTTP servers?
2. What is the main drawback of relying solely on sticky sessions to scale WebSockets?
3. What role does Redis Pub/Sub (or a similar broker) play in scaling WebSocket clusters?
4. Why should presence tracking (who is currently online) use a shared store like Redis rather than an in-memory map on each server?
5. What problem does connection draining during a rolling deployment solve?
Was this page helpful?
You May Also Like
WebSockets Behind a Load Balancer
How to correctly configure load balancers, proxies, and timeouts so WebSocket handshakes and long-lived connections survive in production.
Backpressure in WebSockets
How to detect and handle slow consumers on a WebSocket connection before unbounded buffering leads to memory exhaustion or unresponsive clients.
WebSockets with Redis Pub/Sub
How Redis Pub/Sub lets a cluster of WebSocket servers broadcast messages to clients connected to any instance, and where it falls short.
Related Reading
Related Study Notes in Web Development
Browse all study notesWebAssembly Study Notes
WebAssembly · 30 topics
Web DevelopmentgRPC Study Notes
Protocol Buffers · 30 topics
Web DevelopmentSpring Boot Study Notes
Java · 30 topics
Web DevelopmentFlask Study Notes
Python · 30 topics
Web DevelopmentDjango Study Notes
Python · 30 topics
Web DevelopmentNext.js Study Notes
JavaScript · 30 topics