What is Caching in System Design?
Learn caching — cache hits and misses, LRU eviction, write-through vs write-back and TTL — with examples and system design interview questions.
Expected Interview Answer
Caching is storing copies of frequently accessed data in a fast, temporary layer so future requests are served from it instead of the slower original source, reducing latency and load on the backing store.
When a request finds its data in the cache it is a cache hit; when it does not it is a cache miss and the system fetches from the source and usually stores it for next time. Because a cache is finite, an eviction policy such as LRU (Least Recently Used) decides what to drop when it fills. Write strategies differ: write-through updates the cache and the database together for consistency, while write-back updates the cache first and the database later for speed at the cost of durability risk. A TTL (time-to-live) expires stale entries so the cache does not serve outdated data forever.
- Cuts latency by serving from a fast in-memory layer
- Reduces load on databases and downstream services
- Improves throughput and scalability under read-heavy traffic
AI Mentor Explanation
Caching is like a scorer keeping the current run rate on a whiteboard beside them instead of recomputing it from the full scorebook every ball. Reading the whiteboard is a cache hit; if a needed figure is not there, that is a miss and they recalculate from the book, then jot it down. The board is small, so old numbers get wiped when space runs out — least-recently-used eviction. That fast local copy is exactly what a cache provides.
Step-by-Step Explanation
Step 1
Check the cache
A request first looks in the fast cache layer for its data.
Step 2
Hit or miss
A hit returns instantly; a miss fetches from the source and stores the result.
Step 3
Evict when full
A policy like LRU drops the least recently used entry to free space.
Step 4
Handle writes and staleness
Choose write-through vs write-back and set a TTL to expire stale data.
What Interviewer Expects
- Cache hit vs cache miss explained
- An eviction policy such as LRU
- Write-through vs write-back trade-offs
- Awareness of staleness/TTL and cache invalidation
Common Mistakes
- Ignoring cache invalidation and serving stale data
- Not mentioning any eviction policy
- Confusing write-through and write-back
- Caching data that changes constantly with low reuse
Best Answer (HR Friendly)
“Caching keeps a copy of frequently used data in a fast layer so the system does not repeatedly hit the slower database. It serves repeat requests almost instantly and only falls back to the source when the data is not cached, which makes apps faster and cheaper to run.”
Code Example
async function getUser(id) {
const key = "user:" + id
let user = await cache.get(key) // check cache first
if (user) return user // cache hit
user = await db.findUser(id) // cache miss -> read source
await cache.set(key, user, { ttl: 300 }) // store with 5-min TTL
return user
}Follow-up Questions
- What is the difference between write-through and write-back caching?
- How does an LRU eviction policy decide what to remove?
- What are common cache invalidation strategies?
- Where can caches live — client, CDN, application or database?
MCQ Practice
1. A request whose data is not found in the cache is called a?
A miss means the data was not in the cache, so the system fetches it from the source.
2. The LRU eviction policy removes the entry that was?
LRU evicts the least recently accessed entry, assuming recently used data is likely to be used again.
3. Which write strategy updates the cache and database together for consistency?
Write-through writes to cache and database in the same operation, keeping them consistent at the cost of write latency.
Flash Cards
Cache hit vs miss? — Hit: data found in cache. Miss: not found, fetched from the source.
What is LRU? — Least Recently Used — evicts the entry unused for the longest time.
Write-through? — Updates cache and database together — consistent but slower writes.
Write-back? — Updates cache first, database later — fast but risks data loss on failure.