The Hardest Problem in Caching
Phil Karlton's famous line that there are only two hard things in computer science — cache invalidation and naming things — exists because a cache is, by definition, a second copy of the truth, and every second copy can drift out of sync with the original. Invalidation is the discipline of removing or refreshing a cached value the moment the underlying data changes, rather than waiting for a TTL to expire, so that users never see data that is provably wrong rather than merely a few seconds old.
Cricket analogy: It is like a scoreboard operator who must instantly correct a batter's total the moment the third umpire overturns a boundary call on review, rather than waiting for the next scheduled scoreboard refresh to show the wrong score for several overs.
Explicit Invalidation on Write
The most direct invalidation strategy is to have the write path itself delete or update the relevant Redis keys as part of the same transaction that changes the source of truth — for example, after UPDATE users SET email = ... in the database, immediately issue a DEL user:profile:{id} against Redis so the next read is forced to repopulate from fresh data. This is precise and immediate, but it requires the application to know every cache key that could be affected by a given write, which becomes error-prone as the number of cached views of the same data grows.
Cricket analogy: It is like a scorer manually striking out and rewriting a batter's dismissal type on the paper scoresheet the instant the on-field umpire signals out, rather than waiting for someone to notice the sheet is wrong later.
import redis
r = redis.Redis(decode_responses=True)
def update_user_email(user_id: str, new_email: str) -> None:
db.execute(
"UPDATE users SET email = %s WHERE id = %s",
(new_email, user_id),
)
# explicit invalidation: force the next read to repopulate
r.delete(f"user:profile:{user_id}")
TTL-Based Invalidation
TTL-based invalidation sidesteps the need to track every affected key by simply letting data expire on its own after a fixed window, accepting a bounded amount of staleness (say, up to 60 seconds) in exchange for much simpler application code. This works well for data where slight staleness is acceptable — a homepage's 'trending articles' widget, for instance — but it is the wrong choice for data where correctness matters immediately, such as an account balance or a seat-availability count, because it guarantees a window during which the cache is known to be wrong.
Cricket analogy: It is like a stadium's giant screen showing a 'live' run rate that only refreshes every thirty seconds instead of every ball, which is fine for casual fans in the stands but wrong for a broadcaster's precise graphics overlay.
A common and pragmatic middle ground is to combine both: use a short TTL as a safety net that guarantees eventual correctness even if an explicit invalidation is missed somewhere in the code, while still explicitly deleting keys on write for the common paths so most reads are actually fresh.
Invalidation via Pub/Sub and Tag-Based Grouping
For systems with multiple cache-holding processes — several API servers, each with their own local in-process cache layered in front of Redis — a write on one server needs to tell the others to drop their local copies too, which Redis Pub/Sub handles by broadcasting an invalidation message like PUBLISH cache-invalidate user:42 that every subscribed server listens for and reacts to. A related technique, tag-based invalidation, groups related keys under a shared tag (often a Redis Set holding the member keys) so that a single business event, like a product's category changing, can invalidate every cached view that touched that category in one operation instead of hunting down each key individually.
Cricket analogy: It is like a stadium PA system broadcasting 'DRS overturned' to every replay screen, commentary box, and scoreboard operator simultaneously, so all of them drop their previous call and show the corrected decision at once.
Pub/Sub messages in Redis are fire-and-forget: if a subscriber is disconnected when the invalidation is published, it will never receive that message and will keep serving a stale local cache until it happens to restart or its own TTL expires. For guaranteed delivery, prefer Redis Streams (which persist messages and support consumer groups with acknowledgment) over plain Pub/Sub for anything invalidation-critical.
- Cache invalidation exists because a cache is a second copy of the truth that can drift from the source of record.
- Explicit invalidation on write (DEL the key right after the database update) gives immediate consistency but requires tracking every affected key.
- TTL-based invalidation is simpler but guarantees a window of known staleness, so it is unsuitable for correctness-critical data.
- Combining a short TTL as a safety net with explicit invalidation on the common write paths is a common pragmatic approach.
- Pub/Sub can broadcast invalidation events to multiple servers with local caches, but delivery is not guaranteed for disconnected subscribers.
- Tag-based invalidation groups related keys so one business event can clear every dependent cached view in a single operation.
- Redis Streams are preferred over plain Pub/Sub when invalidation delivery must be guaranteed, since Streams persist messages for consumer groups.
Practice what you learned
1. Why is cache invalidation considered a hard problem?
2. What is the main drawback of explicit invalidation on write?
3. What guarantee does TTL-based invalidation NOT provide?
4. Why might Redis Streams be preferred over Pub/Sub for cache invalidation?
5. What does tag-based invalidation achieve?
Was this page helpful?
You May Also Like
Caching Strategies with Redis
A practical guide to the core caching patterns — cache-aside, write-through, and write-behind — used to speed up applications with Redis, plus how to pick sane TTLs and eviction policies.
Redis as a Session Store
How Redis centralizes web session state across multiple application servers, from modeling sessions as hashes with TTLs to session security and scaling with Sentinel and Cluster.
Pipelining in Redis
How Redis pipelining collapses many network round trips into one for bulk operations, how it differs from MULTI/EXEC transactions, and when it can't help.