Compaction

Log Compaction

How Kafka's compaction cleanup policy retains only the latest value per key instead of expiring records by time.

What is an LSM-Tree and How Does it Handle Writes?

A Log-Structured Merge-tree (LSM-tree) is a storage engine design that turns random writes into fast sequential appends by first buffering writes in an in-memo…

What is Write Amplification and Why Does it Matter?

Write amplification is the ratio between the amount of data actually written to physical storage and the amount of data the application logically intended to w…

What Is Fragmentation?

Fragmentation is the wasted memory that builds up as blocks are allocated and freed over time, leaving usable space scattered in pieces too small or too oddly…

Internal vs External Fragmentation

Internal fragmentation is wasted space inside an allocated block because it is larger than what was requested, while external fragmentation is wasted space bet…

First Fit vs Best Fit vs Worst Fit Memory Allocation

First fit, best fit, and worst fit are contiguous-memory allocation strategies that scan a free-block list differently: first fit picks the first hole large en…

What Causes External Fragmentation and How Is It Solved?

External fragmentation happens when free memory becomes scattered into many small, non-contiguous holes so that even though total free space is sufficient, no…

How Does a Write-Ahead Log Relate to an LSM-Tree?

A write-ahead log provides crash durability for a single write, while an LSM-tree is the broader storage engine architecture that uses that WAL alongside an in…

What is Write Amplification and Why Does It Matter for Storage Engines?

Write amplification is the phenomenon where a single logical write from an application triggers a much larger amount of physical data actually written to durab…

What is Read Amplification in Storage Engines?

Read amplification is the ratio between the amount of data a storage engine actually reads from disk and the amount of data the application logically requested…

What is Space Amplification in Storage Engines?

Space amplification is the ratio between the actual disk space a storage engine consumes and the size of the logical, deduplicated data it is meant to represen…

What is a Log-Structured Merge (LSM) Tree?

A log-structured merge (LSM) tree is a write-optimized storage data structure that buffers writes in an in-memory sorted table and periodically flushes them as…