From One-Liner to Script File
When a text transformation needs more than two or three commands, cramming it into a single shell one-liner becomes unreadable and fragile. The disciplined move is a script file: a plain text file of sed commands, one per line, comments allowed with #, invoked as sed -f clean.sed input. This gives you version control, review, and the ability to reason about a pipeline of edits — deleting comments, normalizing whitespace, and reformatting fields — as a coherent program rather than a wall of semicolons.
Cricket analogy: Graduating to a script file is like a team moving from improvised gully cricket to a planned batting order with defined roles for each position.
The Pattern Space, Hold Space, and Labels
sed processes input line by line into a buffer called the pattern space, where commands operate. For anything beyond single-line edits, sed offers a second buffer, the hold space, plus commands to move data between them: h/H copy or append the pattern space into the hold space, g/G copy or append the hold space back, and x swaps them. Combined with labels (:label) and branch commands (b to jump, t to jump only if a substitution succeeded), these turn sed into a small stateful machine capable of multi-line joins and lookaheads.
Cricket analogy: The hold space is the non-striker's end: you park a batter (a line) there with h, then bring them back on strike with g when the situation needs them.
#!/usr/bin/env -S sed -nf
# clean.sed — strip comments, trim whitespace, join continuation lines
# 1. Skip full-line comments and blank lines
/^[[:space:]]*#/d
/^[[:space:]]*$/d
# 2. Join lines ending in a backslash with the next line
:join
/\\$/ {
N # append next line to pattern space
s/\\\n[[:space:]]*/ / # replace backslash+newline+indent with a space
b join # loop in case of multiple continuations
}
# 3. Trim leading and trailing whitespace, then print
s/^[[:space:]]+//
s/[[:space:]]+$//
pRun this script with: sed -nf clean.sed config.raw > config.clean. The -n suppresses default output so only the explicit p at the end emits lines, and -f loads the commands from the file. Keeping each transformation as a commented block makes the script self-documenting and easy to extend when requirements change.
Looping Safely with Branches
The join block above uses a label :join and an unconditional branch b join to loop while a line still ends in a backslash — this is how sed handles an unknown number of continuation lines. The conditional branch t is subtler: it jumps only if a substitution has succeeded since the last input line or last t, which makes it perfect for 'keep applying this transform until nothing changes' loops. The discipline is ensuring every loop has an exit condition; a branch that always matches will hang sed forever on a line.
Cricket analogy: A branch loop with an exit is like a bowler bowling maidens until the required run rate forces a change of plan — the condition ends the pattern.
An unconditional branch (b label) with no way to change the loop condition will spin forever on the current line, and sed will appear to hang consuming CPU. Always guard branch loops so that each iteration measurably progresses toward an exit — for example, the substitution inside the loop must eventually stop matching. Prefer t (branch-if-substituted) when the loop's purpose is 'repeat until no further change'.
# Collapse multiple spaces into one — loop until no double space remains
sed ':a; s/ / /; ta' file.txt
# Reverse the characters of each line (classic hold-space + loop demo)
sed '/\n/!G; s/\(.\)\(.*\n\)/&\2\1/; //D; s/.//' file.txt- Promote multi-command logic to a -f script file with # comments for readability and version control.
- The pattern space is the per-line working buffer; the hold space is a second buffer for cross-line state.
- h/H, g/G, and x move data between pattern and hold spaces for multi-line processing.
- Labels (:name) plus b (unconditional) and t (branch-if-substituted) give sed loops and conditionals.
- N appends the next input line into the pattern space, enabling multi-line joins and lookaheads.
- Every branch loop must have a guaranteed exit condition or sed will hang on a line.
- The ':a; s/ / /; ta' idiom repeats a substitution until it no longer matches.
Practice what you learned
1. What is the hold space in sed?
2. Which command appends the NEXT input line into the current pattern space?
3. How does the conditional branch t differ from the unconditional b?
4. What does the idiom sed ':a; s/ / /; ta' accomplish?
5. Why is it recommended to move complex sed logic into a -f script file?
Was this page helpful?
You May Also Like
Common sed Idioms
A curated set of battle-tested sed one-liners for deletion, printing ranges, in-line insertion, and squeezing blank lines that you will reach for again and again.
sed Best Practices
Practical habits for writing sed one-liners and scripts that are readable, portable, and safe to run against real data.
sed Quick Reference
A compact, scannable cheat sheet of sed's essential commands, addresses, flags, and regex constructs for fast day-to-day lookup.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics