sed: Stream Editing Basics
sed (Stream EDitor) reads input line by line, applies editing commands you specify, and writes the result to standard output — without ever opening an interactive editor. It's the tool of choice for programmatic text transformation: rewriting config values across many files, stripping unwanted lines from generated output, or performing search-and-replace inside a shell pipeline or script where a human can't be present to click 'find and replace.' Because it operates as a filter, sed composes naturally with pipes and works equally well on a single file or a stream of piped data.
Cricket analogy: sed is like a scorer who reads each ball's outcome line by line and instantly updates the scorecard format without stopping play to open a formal ledger — perfect for rewriting hundreds of scoresheets' formats in one pass without a human editing each by hand.
The substitute command
The workhorse of sed is s/pattern/replacement/flags. By default sed replaces only the first match on each line; the g flag makes it replace every match on the line ('global' within the line, not the whole file — sed still processes every line by default anyway). The i flag (GNU extension) makes matching case-insensitive. You can capture parts of the pattern in parentheses and reference them in the replacement with \1, \2, etc., which is invaluable for reordering or extracting structured text. The delimiter doesn't have to be / — when your pattern itself contains slashes (like a file path), swapping to another delimiter such as s#pattern#replacement# avoids a forest of escaped slashes.
Cricket analogy: s/pattern/replacement/ replaces the first no-ball call on each scoresheet line; the g flag catches every no-ball on that line, the i flag (GNU) ignores case in the umpire's notation, and capturing the bowler's name in parentheses lets you reorder it into 'Bumrah: no-ball' with \1.
# Replace first occurrence of 'foo' with 'bar' on each line
sed 's/foo/bar/' input.txt
# Replace ALL occurrences on each line
sed 's/foo/bar/g' input.txt
# Case-insensitive, in-place edit with a backup (.bak)
sed -i.bak 's/DEBUG/INFO/gi' app.conf
# Use a different delimiter to avoid escaping slashes in paths
sed 's#/var/www/old#/var/www/new#g' nginx.conf
# Delete lines matching a pattern (e.g. comments and blank lines)
sed '/^#/d; /^$/d' /etc/ssh/sshd_config
# Print only a line range (like head/tail combined)
sed -n '10,20p' access.log
# Capture groups: swap 'Last, First' to 'First Last'
echo 'Doe, John' | sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\2 \1/'Editing in place and addressing lines
The -i flag edits files in place rather than printing to stdout; on GNU sed, -i alone edits with no backup, while -i.bak (no space between -i and the suffix) creates a backup with that suffix before editing — note that BSD/macOS sed requires the suffix argument even if empty (-i ''), a common portability trap. Sed commands can be restricted to specific lines using addresses: a line number (3d deletes line 3), a range (2,5d), a regex (/pattern/d), or a regex range (/START/,/END/d deletes everything between two markers, inclusive). The special address $ refers to the last line, useful for operations like sed '$d' to delete the final line of a file.
Cricket analogy: sed -i edits the scoresheet in place with no backup, while -i.bak keeps a carbon copy before overwriting; sed '$d' deletes only the final over's line, and /START/,/END/d strips everything between the rain-delay marker and resumption, inclusive.
GNU sed and BSD sed (shipped by default on macOS) differ meaningfully in flag syntax, most notoriously with -i: GNU sed treats -i.bak as one argument, while BSD sed requires the backup suffix as a separate mandatory argument, so -i '' is needed on BSD/macOS for no-backup in-place edits. Scripts intended to be portable often detect the platform or install GNU sed via Homebrew (gsed) to sidestep this.
Running sed -i directly on a file you haven't backed up, with a regex you haven't tested, is a common way to silently corrupt configuration files — sed will happily 'succeed' even if the pattern matches nothing you intended, or too much. Test first with a plain (non -i) run, or use -i.bak so you always have a rollback.
- sed's core operation is
s/pattern/replacement/flags; add g for all matches per line, i for case-insensitive matching. - -i edits files in place; GNU sed's
-i.bakand BSD sed's-i ''differ in required syntax — a common portability pitfall. - Addresses (line numbers, ranges, regex, or $ for last line) restrict which lines a command applies to.
- Capture groups
\(...\)(BRE) or(...)(ERE with -E) let you reference matched substrings as \1, \2 in the replacement. /^pattern/,/^end/ddeletes everything between two matching markers, inclusive — handy for stripping blocks of text.- Always test sed substitutions without -i first, or keep a backup suffix, before overwriting important files.
Practice what you learned
1. What does the `g` flag do in `sed 's/foo/bar/g' file.txt`?
2. Which sed command deletes all blank lines and lines starting with # from a file's output?
3. Why does `sed -i ''` (with an empty string argument) matter specifically on BSD/macOS sed?
4. In `sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\2 \1/'`, what do \1 and \2 refer to?
5. What does the address range `/START/,/END/d` do in a sed script?
Was this page helpful?
You May Also Like
grep and Regular Expressions
Master text searching with grep and the regular expression syntax that powers it, from basic literal matches to extended regex patterns and useful flags.
awk Basics
Learn awk, the field-oriented text processing language, to extract columns, compute aggregates, and transform structured text like logs and CSVs.
Viewing and Editing Files (cat, less, nano, vim basics)
Learn the core command-line tools for reading and modifying text files, from quick dumps with cat to paginated viewing with less and full editing in nano and vim.
Pipes and Redirection
Learn how the shell wires standard input, output, and error between commands and files using redirection operators and pipes to build powerful command chains.
Related Reading
Related Study Notes in DevOps
Browse all study notesNginx Study Notes
DevOps · 30 topics
DevOpsAnsible Study Notes
DevOps · 30 topics
DevOpsAdvanced Kubernetes Study Notes
Kubernetes · 30 topics
DevOpsAdvanced Bash Scripting Study Notes
Bash · 30 topics
DevOpsApache Kafka Study Notes
Kafka · 30 topics
DevOpsDocker & Kubernetes Study Notes
YAML · 40 topics