What Are Redis Streams?
A Redis Stream is an append-only log data type where each entry gets a unique, monotonically increasing ID like 1657450000000-0 (millisecond timestamp plus sequence number), added via XADD mystream '*' field value. Unlike a List, a Stream lets multiple independent readers consume the same entries at their own pace, and XRANGE/XLEN let you query a range of entries or the total count without removing anything, similar to querying an append-only commit log.
Cricket analogy: Like a ball-by-ball commentary log where each delivery gets a permanent record (over 14, ball 3, six runs) that stays in the scorebook forever, XADD appends a new entry with a unique ID that never gets rewritten.
Reading with XREAD and Consumer Groups
XREAD BLOCK 0 STREAMS mystream $ lets a single consumer poll for new entries, but production systems typically use consumer groups: XGROUP CREATE mystream mygroup $ creates a named group, and XREADGROUP GROUP mygroup consumer1 COUNT 10 STREAMS mystream > delivers unread entries to a specific consumer within that group and tracks them as 'pending' until the consumer calls XACK to confirm processing, giving Kafka-like at-least-once delivery semantics with load-balanced consumers.
Cricket analogy: Like a fielding captain assigning specific balls to specific fielders via hand signals so no catch opportunity is double-covered, XREADGROUP hands each Stream entry to one consumer, and XACK confirms the catch was taken cleanly.
XADD orders * order_id 1042 amount 59.99
XGROUP CREATE orders order-processors $ MKSTREAM
XREADGROUP GROUP order-processors worker-1 COUNT 5 BLOCK 5000 STREAMS orders >
XACK orders order-processors 1657450000000-0Trimming and Retention
Streams grow unbounded unless trimmed: XADD mystream MAXLEN ~ 100000 '*' field value caps the stream near 100,000 entries using the approximate (~) trimming strategy, which is far cheaper than exact trimming because Redis can drop whole macro-nodes of its underlying radix-tree structure instead of trimming to an exact count. Alternatively, XTRIM mystream MAXLEN 100000 or a MINID-based trim removes entries older than a given ID, which is useful for time-based retention windows.
Cricket analogy: Like a stadium archive keeping only the last 100,000 ball-by-ball records and periodically purging the oldest overs in bulk rather than deleting one ball at a time, MAXLEN ~ trims a Stream in bulk chunks for efficiency.
Use approximate trimming (MAXLEN ~ N) in production instead of exact trimming (MAXLEN N without ~) — the exact form forces Redis to trim node-by-node on every single XADD, which is measurably slower under high write throughput.
Common Use Cases and Pitfalls
Streams are commonly used for event sourcing (an immutable log of domain events), activity feeds, and durable work queues where consumer groups provide load balancing with acknowledgment. The main operational pitfall is a consumer crashing after XREADGROUP but before XACK, leaving entries stuck in the Pending Entries List (PEL) — XPENDING shows these, and XCLAIM or XAUTOCLAIM can hand a stuck entry to a different, healthy consumer after an idle-time threshold.
Cricket analogy: Like a review referred to the third umpire that never gets resolved because the broadcast feed drops, leaving the scoreboard stuck, a crashed consumer leaves entries stuck in the PEL until XCLAIM reassigns them.
If a consumer process dies mid-processing, its claimed Stream entries stay in the PEL forever unless another consumer runs XCLAIM/XAUTOCLAIM to reclaim them — monitor XPENDING and build a reclaim loop, or work silently piles up in the pending list while never actually completing.
- Streams are an append-only log type with monotonically increasing IDs, added via XADD.
- XRANGE and XLEN let you query entries and counts without removing them from the Stream.
- Consumer groups (XGROUP CREATE, XREADGROUP) load-balance entries across multiple consumers with at-least-once delivery.
- XACK confirms processing; unacknowledged entries stay in the Pending Entries List (PEL).
- Use MAXLEN ~ (approximate trimming) rather than exact MAXLEN for better write performance at scale.
- XPENDING, XCLAIM, and XAUTOCLAIM recover work from crashed or stalled consumers.
- Streams suit event sourcing, activity feeds, and durable work queues better than Pub/Sub, which drops messages for offline subscribers.
Practice what you learned
1. What command appends a new entry to a Redis Stream?
2. What does XACK do in a consumer group workflow?
3. Why is MAXLEN ~ preferred over exact MAXLEN for trimming in production?
4. What happens to a Stream entry if the consumer that read it crashes before calling XACK?
5. Which command lets a healthy consumer take over entries stuck pending on a crashed consumer?
Was this page helpful?
You May Also Like
Redis with Node.js or Python
How to connect to, query, and manage Redis connections from Node.js (node-redis/ioredis) and Python (redis-py) application code.
Redis Quick Reference
A condensed cheat sheet of the most-used Redis commands across data types, key management, and server administration.
Redis Interview Questions
A study guide covering the Redis concepts most commonly probed in interviews: data structures, persistence, replication, and scaling tradeoffs.