Cache Coherence
Cache coherence is the property of a multiprocessor system in which all processor cores see a consistent, up-to-date view of shared memory data, even though each core maintains its own local cache copies of that data.
Definition
Cache coherence is the property of a multiprocessor system in which all processor cores see a consistent, up-to-date view of shared memory data, even though each core maintains its own local cache copies of that data.
Overview
Modern CPUs give each core its own private cache to avoid the latency of constantly accessing main memory, following the Von Neumann Architecture principle that memory access is a major performance bottleneck. But when multiple cores each cache their own copy of the same memory location, a problem arises: if one core modifies its cached copy, other cores' cached copies become stale unless something actively keeps them synchronized. Cache coherence protocols solve this by ensuring that whenever one core writes to a shared memory location, all other cores are notified or updated so no core ever reads outdated data. The most widely used class of cache coherence protocols is based on the MESI protocol (Modified, Exclusive, Shared, Invalid), which tags each cache line with one of these states to track whether it is uniquely owned, shared read-only across cores, or has been invalidated because another core wrote a newer value. These protocols typically operate over a shared bus or interconnect using either 'snooping' (each cache observes all memory traffic to detect conflicting writes) or a directory-based approach (a central structure tracks which cores hold which cache lines), with directory-based schemes scaling better to systems with many cores. Cache coherence is essential for correct Multithreading and Parallel Computing on shared-memory systems — without it, multi-core programs could silently produce wrong results due to cores operating on inconsistent data. However, maintaining coherence has real performance costs: heavy contention over a shared cache line across many cores, known as 'cache line ping-ponging' or false sharing, can significantly degrade performance in Superscalar Architecture and highly parallel systems, making cache-aware data layout an important consideration in high-performance software design.
Key Concepts
- Ensures all CPU cores see a consistent view of shared memory despite private caches
- Commonly implemented via the MESI protocol tracking Modified, Exclusive, Shared, and Invalid states
- Uses snooping or directory-based approaches to detect and propagate cache line changes
- Essential for correctness in multithreaded programs on shared-memory multiprocessors
- Introduces performance costs from cache line invalidation and synchronization traffic
- False sharing occurs when unrelated data on the same cache line causes unnecessary coherence traffic
- Directory-based protocols scale better than snooping for systems with many cores
- A key design concern in modern multi-core and many-core processor architecture
Use Cases
Frequently Asked Questions
From the Blog
How to cache dependencies in CI without breaking builds
Speed up pipelines safely: designing cache keys from lockfile hashes, choosing what to cache, restore-key fallbacks and spotting a poisoned cache.
Read More Cloud & CybersecurityDesigning a caching layer and avoiding cache stampedes
Add a cache without adding an outage: choosing what to cache, invalidation strategies, TTL jitter, single-flight rebuilds and handling cold starts safely.
Read More AI & TechnologyThe KV cache explained: why it dominates LLM memory during serving
The KV cache makes generation fast and memory-bound at once. Learn what it stores, why it caps concurrency, and the levers that shrink it without hurting quality.
Read More Data ScienceHow to build a tf.data pipeline that stops starving your GPU
Low GPU utilisation usually means the input pipeline cannot keep up. Confirm it by timing the pipeline alone, then order the operations correctly — map, cache, shuffle, batch, prefetch — and parallelise the expensive stages. Ordering affects correctness as well as throughput.
Read More