Designing a Chat Application
A chat application (like WhatsApp, Slack, or iMessage) must deliver messages between users in near real time, preserve ordering within a conversation, work across multiple devices per user, tolerate flaky mobile networks, and scale to millions of concurrent connections. Unlike a typical request/response web service, chat is fundamentally about maintaining long-lived, bidirectional connections and reliably routing events to whichever server currently holds a recipient's connection — this connection-routing problem, along with message delivery guarantees, is what makes chat system design distinct from most CRUD services.
Cricket analogy: A chat app is like live ball-by-ball commentary that must reach every fan's radio in near real time and in the exact order balls were bowled, work whether they're listening on a phone or car radio, and keep broadcasting even through patchy stadium wifi — fundamentally different from a static post-match scorecard page.
Connection management and the routing problem
Clients maintain a persistent WebSocket (or similar, e.g. MQTT for mobile-optimized cases) connection to one of many stateful connection servers behind a load balancer. Because a user's connection lives on exactly one specific server at a time, sending them a message requires knowing which server that is — a stateless HTTP-style 'any server can handle any request' model doesn't directly apply. Systems solve this with a connection registry (commonly Redis) mapping user_id -> connection_server_id, updated on connect/disconnect. When server A needs to deliver a message to a user connected on server B, it looks up the mapping and forwards the message to server B via an internal pub/sub channel or RPC, and server B pushes it down the live socket.
Cricket analogy: Each broadcaster maintains a dedicated live feed to one specific production truck (not just any truck), so routing a director's cue to a specific commentator requires knowing which truck they're plugged into — a central assignment sheet maps commentator to truck, and one truck relays cues to another via an internal radio channel.
Delivery guarantees, ordering, and offline users
Messages are always persisted to durable storage (a message store, often partitioned by conversation ID) before being considered 'sent,' so delivery can be retried if the recipient is offline — the write to storage and the real-time push to a connected recipient happen independently. Ordering within a single conversation is typically preserved using a per-conversation monotonically increasing sequence number, since wall-clock timestamps across distributed servers aren't reliably ordered. For offline users, the system queues undelivered messages and pushes them (or notifies via a mobile push notification service like APNs/FCM) once the user reconnects. Read receipts and delivery receipts are usually separate, asynchronous status updates layered on top of the base message flow, not blocking parts of it.
Cricket analogy: A run is only recorded 'official' once written to the master scorebook, before the crowd sees it flash on the big screen, and deliveries are numbered sequentially per over rather than trusting stadium clocks; a fan who stepped out gets the missed overs queued up and pushed via a recap notification, with 'watched' status tracked separately from the raw commentary feed.
Message send flow (1:1 or small group):
1. Client A sends message over its WebSocket to Connection Server 1
2. Server 1 assigns per-conversation sequence number, writes message to durable Message Store
3. Server 1 publishes event to internal pub/sub: "deliver to user B"
4. Registry lookup: user B's active connection is on Connection Server 2 (or offline)
-> if online: Server 2 receives event, pushes message down B's live socket
-> if offline: message stays queued in Message Store; push notification sent via APNs/FCM
5. Client A receives a "sent" ack once Message Store write succeeds
6. Client B, on receipt, sends a delivery receipt back through the same path (async, non-blocking)
7. When B opens the conversation, client fetches any missed messages by sequence number rangeWhatsApp's original architecture (built on Erlang/OTP) is a well-known real-world example of this pattern: extremely lightweight per-connection processes on connection servers, a message queue/store for durability, and reliance on sequence numbers rather than timestamps for ordering — a design choice that let a small engineering team support hundreds of millions of concurrent connections.
A common mistake is relying on wall-clock timestamps to order messages across devices and servers. Clock skew between distributed servers (and especially client devices, which users can manually change) means two messages sent moments apart can appear out of order or even identical in timestamp. A per-conversation logical sequence number, assigned by a single authoritative point (the server handling the write), avoids this entirely.
- Chat systems require long-lived, stateful connections, unlike typical stateless request/response services.
- A connection registry (e.g., Redis mapping user_id to server_id) is essential for routing messages to the server holding a recipient's live connection.
- Messages are persisted to durable storage before being marked sent, decoupling durability from real-time delivery.
- Per-conversation sequence numbers, not wall-clock timestamps, are the reliable way to preserve message ordering.
- Offline delivery relies on message queuing plus mobile push notifications (APNs/FCM) to wake the client.
- Read/delivery receipts are layered asynchronously on top of the core message flow and should not block message delivery.
Practice what you learned
1. Why can't a stateless 'any server handles any request' model be applied directly to chat message delivery?
2. What is the purpose of the connection registry (e.g., a Redis map of user_id to connection_server_id)?
3. Why do chat systems use per-conversation sequence numbers instead of wall-clock timestamps for ordering?
4. How are messages typically delivered to a user who is currently offline?
5. Why are read receipts and delivery receipts typically implemented as separate, asynchronous updates rather than part of the core send path?
Was this page helpful?
You May Also Like
Designing a News Feed System
Explores how systems like Facebook or Twitter assemble a personalized, ranked feed at scale, contrasting fan-out-on-write and fan-out-on-read delivery models.
Pub/Sub Architecture
Publish-subscribe architecture decouples message producers from an unknown, dynamic set of consumers by routing messages through named topics rather than direct point-to-point links.
WebSockets and Long Polling
Compares techniques for pushing real-time updates from server to client, from wasteful short polling through long polling to full-duplex WebSocket connections.
Message Queues Explained
Message queues decouple producers from consumers by buffering work as discrete messages, enabling asynchronous processing, load leveling, and resilience to downstream slowdowns.
Related Reading
Related Study Notes in Software Engineering
Browse all study notesMicroservices Study Notes
Software Architecture · 30 topics
Software EngineeringTesting & TDD Study Notes
Software Testing · 30 topics
Software EngineeringDesign Patterns Study Notes
Software Design · 30 topics
Software EngineeringSoftware Engineering Study Notes
Python · 75 topics
Software EngineeringGit & Version Control Study Notes
Bash · 40 topics