Streams and buffers — Node.js & Express Backend

Every Node.js application works with data — reading files, receiving HTTP request bodies, writing to databases, sending responses. The naive approach to all of this is to load data into memory as a complete unit, process it, and write it out. For small payloads this works fine, but it breaks catastrophically at scale: reading a 4GB log file into memory crashes your server, piping a video stream through your API without streaming means your users wait indefinitely for the entire file to buffer before playback begins, and processing thousands of database records at once exhausts your heap. Streams solve this by making data processing incremental — data flows through your application in small chunks, each chunk processed and released before the next arrives. Buffers are the lower-level primitive that holds the raw bytes between those chunks. Together, streams and buffers are the foundation of every high-throughput, memory-efficient Node.js application, from HTTP servers to file processors to real-time data pipelines. Understanding them is non-negotiable for production-grade backend work.

Analogy🏏Cricket

🏏 Think of it like cricket: In a Test match, there are multiple types of scheduled breaks and interruptions, each with different priority levels and timing rules. The lunch break (setTimeout) is scheduled for a specific time but can be delayed if a wicket falls just before — it fires at the next available opportunity after its scheduled time, not at the exact scheduled time. The drinks break (setImmediate) is taken at the end of the current over, as soon as the ongoing delivery sequence finishes. The umpire's over-rate check (process.nextTick) happens immediately after each delivery, before the next batsman faces — it cannot be deferred. The DRS review resolution (Promise microtask) happens between deliveries, after the umpire check but before any scheduled breaks. Just as the match referee would be furious if an umpire kept conducting over-rate checks after every single ball indefinitely (starving nextTick), the event loop is unable to proceed if process.nextTick callbacks keep registering new nextTick callbacks recursively. The insight is that each timer primitive is not just a delay mechanism — it is a specific position in a formal protocol, and misusing it violates that protocol's invariants.