Advanced Redirection

Analogy categoryAnalogies

Beyond > and <: File Descriptors

Every process starts with three open file descriptors: 0 (stdin), 1 (stdout), and 2 (stderr). Bash lets you duplicate, close, and reopen these descriptors independently using the N>&M syntax, and the exec builtin lets you retarget a descriptor for the rest of a script or block rather than for just one command, which is essential once a script needs to log, prompt, and process data simultaneously.

🏏

Cricket analogy: Just as a stadium has separate feeds for the broadcast commentary, the stump microphone, and the giant screen replay, a bash process keeps stdin, stdout, and stderr as three independent channels you can route to different destinations like MS Dhoni directing fielders to three different positions at once.

Merging stdout and stderr

The order of redirection operators matters because bash processes them strictly left to right: cmd > file 2>&1 sends stdout to file and then points stderr at wherever stdout currently points (the file), so both end up in file. But cmd 2>&1 > file first points stderr at the terminal (wherever stdout was at that moment) and only then sends stdout to file, leaving stderr still going to the terminal. The bash-specific shorthand &>file merges both streams into one file in a single, unambiguous step.

🏏

Cricket analogy: If a scorer first copies the current total onto a duplicate scoreboard and then updates the main scoreboard, the duplicate still shows the old total, just as 2>&1 > file captures stderr's destination before stdout is repointed to the file.

bash

# WRONG ORDER: stderr goes to terminal, only stdout goes to build.log
./compile.sh 2>&1 > build.log

# RIGHT ORDER: stdout goes to build.log, then stderr is duplicated onto it too
./compile.sh > build.log 2>&1

# Bash shorthand: unambiguous, merges both in one step
./compile.sh &> build.log

# Send stdout to a log and stderr to a separate error log, both at once
./compile.sh > build.log 2> build.err

# Tee stdout to both the terminal and a file while still checking exit status
./compile.sh 2>&1 | tee build.log

Process Substitution

Process substitution, written <(command) or >(command), lets bash treat a running command's output (or input) as if it were a regular file by creating a temporary named pipe or /dev/fd/N entry. This is invaluable when a tool only accepts filenames as arguments but you want to feed it the live output of another command, such as diffing two command outputs directly, or piping into multiple parallel consumers with tee >(cmd1) >(cmd2) without ever writing an intermediate temp file to disk.

🏏

Cricket analogy: Instead of writing both teams' innings scorecards to paper and comparing them later, a stats analyst compares two live scoreboard feeds side by side in real time, just as diff <(sort a.txt) <(sort b.txt) compares two live command outputs without intermediate files.

diff <(sort file1.txt) <(sort file2.txt) is a classic use of process substitution: it lets you diff two sorted versions of files without ever creating a physical temp file, and bash cleans up the underlying /dev/fd entry automatically when the command finishes.

noclobber, /dev/null, and /dev/tty

set -o noclobber prevents the > operator from silently overwriting an existing file, forcing you to use >| to force truncation deliberately; this guards a script author against destroying data with a mistyped redirect target. Redirecting to /dev/null discards output entirely, which is the standard way to silence a noisy command's stdout, stderr, or both (> /dev/null 2>&1), while /dev/tty always refers to the controlling terminal regardless of any other redirection in effect, letting a script prompt the interactive user even when its own stdout has been redirected to a log file.

🏏

Cricket analogy: A groundsman who requires a supervisor's explicit sign-off before repainting the pitch markings, rather than letting anyone overwrite them casually, mirrors how noclobber blocks accidental overwrites unless you explicitly force it with >|.

noclobber is NOT enabled by default in bash, so plain > will silently truncate an existing file unless you explicitly run set -o noclobber yourself; scripts that write important output should either check for the file's existence first or enable noclobber deliberately.

A process has three default file descriptors: 0 (stdin), 1 (stdout), 2 (stderr), each independently redirectable.
Redirection order matters: > file 2>&1 merges stderr into the file, but 2>&1 > file does not.
&>file is bash shorthand that unambiguously merges stdout and stderr into one file.
Process substitution <(cmd) and >(cmd) exposes a command's live output/input as a pseudo-file without writing temp files.
> /dev/null 2>&1 is the standard idiom for silencing a command's output entirely.
/dev/tty always targets the controlling terminal, useful for prompting users even when stdout is redirected.
set -o noclobber protects existing files from accidental overwrite by >, requiring >| to force truncation.