Stream Processing

What is a Count-Min Sketch and When Would You Use One?

A count-min sketch is a probabilistic data structure that estimates the frequency of items in a massive stream using a fixed-size 2D array of counters and seve…

How to Design an Ad Click Aggregator

An ad click aggregator ingests a high-throughput stream of click events, deduplicates and validates them, then rolls them up into per-ad, per-time-window count…

How Would You Design a Top-K Trending System?

A top-K trending system continuously ranks items (like hashtags or search terms) by recent activity using a streaming pipeline that maintains approximate count…

How to Design a Real-Time Analytics Dashboard?

A real-time analytics dashboard is designed around a streaming ingestion pipeline that continuously aggregates raw events into pre-computed rollups at multiple…

What is a Count-Min Sketch and How Does It Estimate Frequencies?

A Count-Min Sketch is a probabilistic data structure that estimates how many times each item has appeared in a stream using a small, fixed amount of memory, at…

What is HyperLogLog and How Does It Count Distinct Elements?

HyperLogLog is a probabilistic algorithm that estimates the number of distinct elements (cardinality) in a massive dataset using only a few kilobytes of memory…

Batch vs Stream Processing: How Do You Choose?

Batch processing runs computations over a bounded, already-collected chunk of data on a schedule, trading latency for simplicity and easy correctness, while st…

What is Lambda Architecture in System Design?

Lambda architecture is a data-processing pattern that runs a slow, accurate batch layer over the complete historical dataset alongside a fast speed layer over…

What is Kappa Architecture and How Does It Differ from Lambda?

Kappa architecture simplifies Lambda architecture by processing all data — historical and real-time — through a single stream-processing pipeline, treating the…

What is Windowing in Stream Processing?

Windowing is the technique of grouping an unbounded stream of events into finite, time-bounded or count-bounded buckets so that aggregations like counts, sums,…

What Are Message Ordering Guarantees?

Message ordering guarantees define whether and how consumers are assured to see messages in the same sequence producers sent them, ranging from no ordering at…