Why Caching Matters
Caching is the practice of storing a copy of data somewhere faster or closer to the consumer than its original source, so that repeated requests for the same data can be served without redoing the expensive work of computing it or fetching it from a slower system. It's one of the highest-leverage techniques in system design because most workloads are read-heavy and skewed — a small fraction of data (a trending video, a popular product page, a hot database row) accounts for a disproportionate share of requests. Serving those hot reads from a cache instead of the database or an expensive computation dramatically cuts both latency and load on the underlying system, often by an order of magnitude or more.
Cricket analogy: Like keeping a star batsman's favorite bat pre-taped and ready in the dugout instead of preparing it fresh every innings, caching stores the hot item close at hand so the most-requested resource (a trending highlight reel) doesn't need to be refetched from the archive every time.
What Makes Data Cacheable
Not all data benefits equally from caching. Good candidates are read frequently relative to how often they change, tolerate some staleness (the business can accept that a cached view might be a few seconds or minutes out of date), and are expensive to produce relative to how cheap it is to store and retrieve them. Data that changes on every read, must always be perfectly fresh (e.g. a real-time account balance before executing a trade), or is accessed so rarely that caching it wastes memory without meaningfully reducing load, are poor caching candidates. The cache hit ratio — the fraction of requests served from cache rather than the underlying source — is the primary metric for whether a cache is earning its keep; a low hit ratio means you're paying the memory and complexity cost without much benefit.
Cricket analogy: Like caching yesterday's completed match scorecard (it never changes and is checked often) versus refusing to cache the live run-rate during an over in progress (it changes every ball and must always be fresh), good caching candidates are stable and frequently read, while volatile, must-be-fresh data isn't.
Where Caches Live
Caching happens at multiple layers of a system simultaneously: browser caches store static assets on the client; CDNs cache content at edge locations near users; a shared in-memory cache (Redis, Memcached) sits between the application and the database to absorb repeated reads; and even the database itself maintains internal buffer pools and query caches. Each layer trades off differently — closer-to-the-user caches (browser, CDN) cut network latency and offload the entire backend, while application-layer caches (Redis) reduce database load but still require a network hop. Layering these caches means a request can be satisfied by the cheapest available layer before falling through to more expensive ones.
Cricket analogy: Like a fan first checking their own memory of a famous score (browser), then the stadium's big screen replay (CDN), then calling the official scorer's desk (Redis) before finally digging through the archive records (database), each layer is checked cheapest-first before falling through to the next.
Request flow with layered caching:
Client -> Browser cache (hit? return instantly)
-> CDN edge cache (hit? return, no origin traffic)
-> App server -> Redis/Memcached cache (hit? skip DB)
|
(miss) v
Database (source of truth)
Each layer absorbs a fraction of traffic; only true cache misses
reach the database, which is usually the most expensive and
hardest-to-scale component.Twitter's timeline-serving architecture relies heavily on caching pre-computed home timelines in memory (fan-out-on-write) precisely because recomputing a user's feed from scratch on every page load — joining recent tweets from everyone they follow — would be far too expensive to do at read time for hundreds of millions of users.
A common mistake is caching data that changes frequently relative to how often it's read, which produces a low hit ratio while still paying the cost of cache invalidation logic and the risk of serving stale data. Caching should be applied where the read-to-write ratio and staleness tolerance actually justify it — not indiscriminately on every data access path.
- Caching stores data closer to the consumer to avoid repeating expensive fetches or computations.
- Most real workloads are read-heavy and skewed, making a small hot dataset responsible for most traffic.
- Good caching candidates are read often, change rarely, and tolerate some staleness.
- Cache hit ratio is the key metric for whether a cache is delivering value.
- Caching happens at multiple layers: browser, CDN, application cache, and database buffer pools.
- Caching data with a low read-to-write ratio wastes complexity and risks staleness without meaningful benefit.
Practice what you learned
1. Why is caching such a high-leverage technique in most system designs?
2. What characteristic makes data a poor candidate for caching?
3. What does cache hit ratio measure?
4. In a layered caching architecture (browser, CDN, application cache, database), what happens on a cache miss at one layer?
5. What is a consequence of caching data with a low read-to-write ratio?
Was this page helpful?
You May Also Like
Cache Eviction Policies
Covers the algorithms caches use to decide what to remove when full, comparing LRU, LFU, FIFO, and TTL-based approaches and their tradeoffs.
Cache Invalidation Strategies
Explores how systems keep cached data consistent with the source of truth, covering TTLs, write-through/write-behind patterns, and explicit invalidation on updates.
CDN Caching and Edge Computing
Covers how Content Delivery Networks cache content close to users to cut latency, and how edge computing extends this to running logic near the request origin.
Database Replication
Explains how copying data across multiple database nodes improves read throughput and fault tolerance, and the consistency tradeoffs of leader-follower and multi-leader setups.
Related Reading
Related Study Notes in Software Engineering
Browse all study notesMicroservices Study Notes
Software Architecture · 30 topics
Software EngineeringTesting & TDD Study Notes
Software Testing · 30 topics
Software EngineeringDesign Patterns Study Notes
Software Design · 30 topics
Software EngineeringSoftware Engineering Study Notes
Python · 75 topics
Software EngineeringGit & Version Control Study Notes
Bash · 40 topics