Designing a News Feed System
A news feed system aggregates content from the people, pages, or topics a user follows into a single, roughly-chronological or ranked timeline. The design challenge is not any single component but the interaction of three demanding requirements simultaneously: extremely high write fan-out (a celebrity account with 50 million followers publishing one post potentially needs to reach 50 million feeds), extremely low read latency (a feed must render in well under a second), and personalization/ranking (the feed is rarely pure chronological order in modern systems — it's scored by predicted engagement). These pull in different directions, which is why the delivery model — fan-out-on-write vs fan-out-on-read — is the central design decision.
Cricket analogy: Like a broadcaster needing to instantly push live commentary to millions of fans following Virat Kohli while also serving a fast-loading scorecard app and ranking key moments like sixes above dot balls.
Fan-out-on-write (push model)
When a user posts, the system immediately writes that post's ID into the precomputed feed (usually a Redis list or similar) of every follower. Reading a feed then becomes a cheap operation — fetch the user's own pre-materialized feed list, which is already sorted and ready to render. The cost is shifted to write time: one post from a user with millions of followers triggers millions of feed insertions, and if that happens synchronously it would make posting unacceptably slow, so it's done asynchronously through a message queue and worker pool. This model works well for the common case (most users have a modest follower count) but breaks down for high-fan-out accounts.
Cricket analogy: Like a stadium PA system writing 'FOUR!' onto every scoreboard the instant it happens, but when Sachin Tendulkar bats, updating every board synchronously would jam the system, so it's queued through runners instead.
Fan-out-on-read (pull model) and the hybrid approach
In the pull model, a user's feed is computed at read time by fetching recent posts from everyone they follow and merging/ranking them on the fly. This avoids the write amplification problem entirely — posting is O(1) regardless of follower count — but shifts cost to read time, and reads are far more frequent than writes for most users, so this alone doesn't scale either. Production systems (Twitter/X is the canonical public example) use a hybrid: fan-out-on-write for the vast majority of users, but celebrity/high-follower accounts are excluded from push fan-out and instead merged in at read time for the small number of users who follow them. This bounds the worst-case write cost while keeping typical reads cheap.
Cricket analogy: Like a fan checking multiple live scoreboards and merging them into one view when they open the app, cheap for casual fans but for a star like Kohli followed by millions, his scores are merged in at read time instead of pushed.
Hybrid fan-out request flow:
WRITE PATH (regular user posts):
1. User posts -> written to Post Store, post_id generated
2. Post event published to a message queue (Kafka)
3. Fan-out workers consume event, look up follower list
4. For each follower: push post_id into their precomputed Feed Cache (Redis list, capped length)
-> if follower count > threshold (e.g. 1M), SKIP push (celebrity exception)
READ PATH (user opens feed):
1. Fetch user's precomputed Feed Cache (fast, already-pushed posts)
2. Fetch recent posts from any celebrity accounts the user follows (pull, small set)
3. Merge both lists, re-rank by engagement-prediction score
4. Hydrate post_ids into full post content (batch fetch from Post Store / CDN for media)
5. Return top N to clientTwitter's engineering team has publicly described exactly this hybrid model: regular tweets fan out to precomputed timelines held in Redis, while accounts with very large follower counts are excluded from fan-out and merged in at read time — a concrete illustration of using a threshold to route two different accounts through two different architectures for the same feature.
A common mistake is treating ranking as an afterthought bolted onto a purely chronological pipeline. In practice, ranking (via a machine-learned model scoring predicted engagement) needs to be part of the core read path design from the start, because it changes what data must be available at read time (recency, author affinity, content type, past engagement signals) — retrofitting it later often requires reworking the feed cache's data model.
- Fan-out-on-write precomputes feeds at post time, making reads cheap but writes expensive for high-follower accounts.
- Fan-out-on-read computes feeds at request time, making writes cheap but reads expensive at scale.
- Production systems use a hybrid: push for typical users, pull for celebrity/high-fan-out accounts merged in at read time.
- Fan-out is performed asynchronously via a message queue and worker pool so posting latency stays low regardless of follower count.
- Modern feeds are ranked by predicted engagement, not pure chronology, which affects what data the feed cache must store.
- The feed cache typically stores post IDs (not full content) with a capped length, hydrating full content from a separate post store at read time.
Practice what you learned
1. What is the primary drawback of a pure fan-out-on-write (push) model for a news feed?
2. Why do production systems commonly use a hybrid fan-out model rather than pure push or pure pull?
3. Why is fan-out-on-write typically performed asynchronously via a message queue rather than synchronously during the post request?
4. What do feed caches typically store to keep them lightweight?
5. Why does modern feed ranking need to be considered early in the system design rather than added later?
Was this page helpful?
You May Also Like
Designing a Chat Application
Covers the architecture behind real-time messaging apps: persistent connections, message delivery guarantees, ordering, presence, and multi-device synchronization.
Pub/Sub Architecture
Publish-subscribe architecture decouples message producers from an unknown, dynamic set of consumers by routing messages through named topics rather than direct point-to-point links.
Message Queues Explained
Message queues decouple producers from consumers by buffering work as discrete messages, enabling asynchronous processing, load leveling, and resilience to downstream slowdowns.
Cache Eviction Policies
Covers the algorithms caches use to decide what to remove when full, comparing LRU, LFU, FIFO, and TTL-based approaches and their tradeoffs.
Related Reading
Related Study Notes in Software Engineering
Browse all study notesMicroservices Study Notes
Software Architecture · 30 topics
Software EngineeringTesting & TDD Study Notes
Software Testing · 30 topics
Software EngineeringDesign Patterns Study Notes
Software Design · 30 topics
Software EngineeringSoftware Engineering Study Notes
Python · 40 topics
Software EngineeringGit & Version Control Study Notes
Bash · 40 topics