Beyond > and <: File Descriptors
Every process starts with three open file descriptors: 0 (stdin), 1 (stdout), and 2 (stderr). Bash lets you duplicate, close, and reopen these descriptors independently using the N>&M syntax, and the exec builtin lets you retarget a descriptor for the rest of a script or block rather than for just one command, which is essential once a script needs to log, prompt, and process data simultaneously.
Cricket analogy: Just as a stadium has separate feeds for the broadcast commentary, the stump microphone, and the giant screen replay, a bash process keeps stdin, stdout, and stderr as three independent channels you can route to different destinations like MS Dhoni directing fielders to three different positions at once.
Merging stdout and stderr
The order of redirection operators matters because bash processes them strictly left to right: cmd > file 2>&1 sends stdout to file and then points stderr at wherever stdout currently points (the file), so both end up in file. But cmd 2>&1 > file first points stderr at the terminal (wherever stdout was at that moment) and only then sends stdout to file, leaving stderr still going to the terminal. The bash-specific shorthand &>file merges both streams into one file in a single, unambiguous step.
Cricket analogy: If a scorer first copies the current total onto a duplicate scoreboard and then updates the main scoreboard, the duplicate still shows the old total, just as 2>&1 > file captures stderr's destination before stdout is repointed to the file.
# WRONG ORDER: stderr goes to terminal, only stdout goes to build.log
./compile.sh 2>&1 > build.log
# RIGHT ORDER: stdout goes to build.log, then stderr is duplicated onto it too
./compile.sh > build.log 2>&1
# Bash shorthand: unambiguous, merges both in one step
./compile.sh &> build.log
# Send stdout to a log and stderr to a separate error log, both at once
./compile.sh > build.log 2> build.err
# Tee stdout to both the terminal and a file while still checking exit status
./compile.sh 2>&1 | tee build.logProcess Substitution
Process substitution, written <(command) or >(command), lets bash treat a running command's output (or input) as if it were a regular file by creating a temporary named pipe or /dev/fd/N entry. This is invaluable when a tool only accepts filenames as arguments but you want to feed it the live output of another command, such as diffing two command outputs directly, or piping into multiple parallel consumers with tee >(cmd1) >(cmd2) without ever writing an intermediate temp file to disk.
Cricket analogy: Instead of writing both teams' innings scorecards to paper and comparing them later, a stats analyst compares two live scoreboard feeds side by side in real time, just as diff <(sort a.txt) <(sort b.txt) compares two live command outputs without intermediate files.
diff <(sort file1.txt) <(sort file2.txt) is a classic use of process substitution: it lets you diff two sorted versions of files without ever creating a physical temp file, and bash cleans up the underlying /dev/fd entry automatically when the command finishes.
noclobber, /dev/null, and /dev/tty
set -o noclobber prevents the > operator from silently overwriting an existing file, forcing you to use >| to force truncation deliberately; this guards a script author against destroying data with a mistyped redirect target. Redirecting to /dev/null discards output entirely, which is the standard way to silence a noisy command's stdout, stderr, or both (> /dev/null 2>&1), while /dev/tty always refers to the controlling terminal regardless of any other redirection in effect, letting a script prompt the interactive user even when its own stdout has been redirected to a log file.
Cricket analogy: A groundsman who requires a supervisor's explicit sign-off before repainting the pitch markings, rather than letting anyone overwrite them casually, mirrors how noclobber blocks accidental overwrites unless you explicitly force it with >|.
noclobber is NOT enabled by default in bash, so plain > will silently truncate an existing file unless you explicitly run set -o noclobber yourself; scripts that write important output should either check for the file's existence first or enable noclobber deliberately.
- A process has three default file descriptors: 0 (stdin), 1 (stdout), 2 (stderr), each independently redirectable.
- Redirection order matters:
> file 2>&1merges stderr into the file, but2>&1 > filedoes not. &>fileis bash shorthand that unambiguously merges stdout and stderr into one file.- Process substitution
<(cmd)and>(cmd)exposes a command's live output/input as a pseudo-file without writing temp files. > /dev/null 2>&1is the standard idiom for silencing a command's output entirely./dev/ttyalways targets the controlling terminal, useful for prompting users even when stdout is redirected.set -o noclobberprotects existing files from accidental overwrite by>, requiring>|to force truncation.
Practice what you learned
1. Which command correctly redirects both stdout and stderr into the same file, build.log?
2. What does `diff <(sort a.txt) <(sort b.txt)` accomplish that piping alone could not?
3. What is the effect of `set -o noclobber` in a script?
4. Why would a script redirect a prompt to /dev/tty instead of letting it go to stdout?
5. What does `command > /dev/null 2>&1` do?
Was this page helpful?
You May Also Like
Here-Documents and Here-Strings
Learn how to feed multi-line literal text into commands using heredocs (<<) and here-strings (<<<), including quoting rules and indentation control.
Advanced awk Scripting
Go beyond simple field printing to use awk's associative arrays, user-defined functions, BEGIN/END blocks, and multiple field separators for real text-processing programs.
Combining grep, awk, and sed
Learn how to chain grep, awk, and sed into efficient pipelines, choosing the right tool for filtering, transforming, and extracting text at each stage.
Related Reading
Related Study Notes in DevOps
Browse all study notesNginx Study Notes
DevOps · 30 topics
DevOpsAnsible Study Notes
DevOps · 30 topics
DevOpsAdvanced Kubernetes Study Notes
Kubernetes · 30 topics
DevOpsApache Kafka Study Notes
Kafka · 30 topics
DevOpsDocker & Kubernetes Study Notes
YAML · 40 topics
DevOpsCI/CD Tools & Pipelines Study Notes
YAML · 37 topics