What are Streams in Node.js?
Understand Node.js streams: Readable, Writable, Duplex, and Transform, how pipe and backpressure work, and why they keep memory flat on huge files.
Expected Interview Answer
Streams are Node.js abstractions for processing data piece by piece (in chunks) as it becomes available, instead of loading an entire resource into memory at once.
There are four stream types: Readable (source), Writable (destination), Duplex (both), and Transform (a duplex that modifies chunks in transit, like compression). All streams are EventEmitters and emit events such as 'data', 'end', and 'error'. The .pipe() method connects a readable to a writable, automatically managing backpressure so a fast source does not overwhelm a slow consumer. This chunked model keeps memory usage flat and constant even for huge files or network payloads.
- Constant, low memory usage regardless of data size
- Start processing before all data arrives (time efficiency)
- Composable via .pipe() and pipeline()
- Automatic backpressure handling
- Ideal for files, HTTP, and network I/O
AI Mentor Explanation
Streams are like watching a match ball by ball instead of waiting for the entire innings to finish before you learn anything. Each delivery arrives as a chunk you can react to immediately, and you never need the whole day's play in your head at once — you process each ball, update the score, and move on, keeping your attention steady no matter how long the game runs.
Step-by-Step Explanation
Step 1
Pick the stream type
Readable for sources, Writable for destinations, Transform for on-the-fly modification, Duplex for both directions.
Step 2
Consume in chunks
Listen for 'data' events or use for-await-of on async iterable readables to process each chunk.
Step 3
Handle lifecycle events
Watch for 'end' when the source is exhausted and always attach an 'error' handler.
Step 4
Connect with pipe
Use source.pipe(destination) to wire streams together and move data automatically.
Step 5
Prefer pipeline()
Use stream.pipeline() to chain streams with proper error propagation and cleanup.
Step 6
Respect backpressure
Let .write() return value and 'drain' events pace how fast you push data to a slow writable.
What Interviewer Expects
- Names the four stream types and their roles
- Explains chunked processing versus buffering everything
- Understands .pipe() and backpressure
- Knows streams are EventEmitters with 'data'/'end'/'error'
- Can give a real use case like reading a large file or HTTP
- Mentions pipeline() for safe error handling
Common Mistakes
- Confusing Transform streams with Duplex streams
- Ignoring backpressure and overwhelming the writable side
- Forgetting to handle the 'error' event, leaking resources
- Using fs.readFile for huge files instead of a read stream
- Chaining raw .pipe() without error handling instead of pipeline()
Best Answer (HR Friendly)
“Streams let Node.js work with data in small pieces as it flows, instead of loading everything into memory first. It is like watching a video that plays while it downloads, which keeps the app fast and memory-light even with very large files.”
Code Example
const fs = require('fs')
const { pipeline } = require('stream')
const { Transform } = require('stream')
// Transform stream: uppercase each chunk as it passes
const upper = new Transform({
transform(chunk, encoding, callback) {
callback(null, chunk.toString().toUpperCase())
}
})
// pipeline wires streams and handles errors + cleanup
pipeline(
fs.createReadStream('input.txt'),
upper,
fs.createWriteStream('output.txt'),
(err) => {
if (err) console.error('Pipeline failed:', err.message)
else console.log('Done — memory stays flat regardless of file size')
}
)Follow-up Questions
- What is backpressure and how do streams handle it?
- How does a Transform stream differ from a Duplex stream?
- Why prefer pipeline() over chained .pipe() calls?
- How do you consume a readable stream with async iteration?
- When would you choose a stream over fs.readFile?
MCQ Practice
1. Which stream type both reads and modifies data as it passes through?
A Transform stream is a Duplex stream whose output is computed from its input, e.g. compression or encryption.
2. What problem does backpressure solve?
Backpressure pauses the readable when the writable's buffer is full, preventing memory blowups.
3. Why is stream.pipeline() preferred over manual .pipe() chains?
pipeline() propagates errors across the whole chain and destroys streams on failure, avoiding leaks.
Flash Cards
Name the four stream types. — Readable, Writable, Duplex, and Transform.
What does .pipe() do? — Connects a readable to a writable and moves data, managing backpressure automatically.
What is backpressure? — A mechanism that slows a fast source when a slow destination's buffer fills up.
Why use pipeline()? — It chains streams with proper error propagation and automatic cleanup.
Are streams EventEmitters? — Yes — they emit 'data', 'end', 'error', and 'drain' events.