AWK's String Toolkit
Text processing is AWK's core purpose, so it ships with a rich set of built-in string functions. The essentials are length(s) for the character count, substr(s, start, len) for extracting a substring, index(s, t) for finding the position of t within s, and split(s, arr, sep) for breaking a string into an array. AWK uses 1-based indexing throughout — the first character is position 1, not 0 — which trips up programmers arriving from C or Python. Mastering these primitives lets you slice and inspect fields without spawning external tools.
Cricket analogy: substr extracting characters is like isolating a specific passage of an innings — 'balls 10 through 20' — pulling out just that segment from the full scorecard, the way substr slices a string.
Searching and Substituting: sub, gsub, match
For pattern-based editing, sub(regex, replacement, target) replaces the first match of a regular expression and returns the number of substitutions made (0 or 1), while gsub(regex, replacement, target) replaces *all* matches and returns the count. Both modify target in place — if omitted, target defaults to $0, the whole record. The match(s, regex) function finds where a regex matches and sets the special variables RSTART (starting position) and RLENGTH (matched length, or -1 if no match), which you can feed into substr to extract the matched text.
Cricket analogy: sub replacing the first match is like the third umpire correcting a single mis-recorded run; gsub is like re-scoring every wide in the innings at once — one fix versus a sweep of all matches.
Formatting and Case: sprintf, toupper, tolower
sprintf(format, ...) builds a formatted string using the same conversion specifiers as printf — such as %-10s for left-aligned text or %05.2f for a zero-padded fixed-point number — but returns the result instead of printing it, so you can assign or further process it. toupper(s) and tolower(s) return case-converted copies without modifying the original. A frequent pattern is normalising keys before counting, e.g. count[tolower($1)]++, so that 'Apple' and 'apple' aggregate together rather than forming two separate buckets.
Cricket analogy: sprintf is like preparing a neatly aligned scorecard line — name padded to a fixed width, average to two decimals — a formatted string built and stored rather than announced aloud.
# extract, search, and reformat fields
awk '{
n = length($0) # total characters in the record
pos = index($1, "@") # position of @ in an email-like field
if (pos > 0)
user = substr($1, 1, pos - 1) # everything before the @
# replace all tabs with a single space in the whole record
gsub(/\t/, " ")
# pull out the first number using match + RSTART/RLENGTH
if (match($0, /[0-9]+/))
num = substr($0, RSTART, RLENGTH)
# build a formatted, normalised summary string
line = sprintf("%-15s len=%d num=%s", tolower(user), n, num)
print line
# split a CSV field into parts
m = split($2, parts, ",")
for (i = 1; i <= m; i++) count[tolower(parts[i])]++
}
END { for (k in count) print k, count[k] }' contacts.txtmatch(s, regex) sets RSTART and RLENGTH as a side effect. Combine them with substr(s, RSTART, RLENGTH) to capture exactly the matched text — a portable way to extract a regex match in POSIX AWK before GNU AWK's match(s, regex, arr) third-argument capture form.
AWK string indexing is 1-based, and substr(s, start, len) clamps out-of-range arguments rather than erroring. substr(s, 0) or a negative start behaves as if starting at position 1, and a length past the end simply returns what remains. These silent adjustments can mask off-by-one bugs — verify your start positions.
- Core functions:
length,substr,index, andsplitcover measuring, slicing, searching, and tokenizing. - AWK string positions are 1-based — the first character is at index 1.
subreplaces the first regex match;gsubreplaces all; both return the count and default to $0.match(s, regex)sets RSTART and RLENGTH; pair withsubstrto extract the matched text.sprintf(fmt, ...)returns a formatted string instead of printing it.toupperandtolowerreturn case-converted copies without altering the original.- Normalise case (e.g.
tolower($1)) before using strings as array keys to aggregate correctly.
Practice what you learned
1. What does `substr("hello", 2, 3)` return in AWK?
2. What is the difference between `sub` and `gsub`?
3. After a successful `match(s, /re/)`, which variables hold the match position and length?
4. How does `sprintf` differ from `printf` in AWK?
5. Why use `count[tolower($1)]++` instead of `count[$1]++`?
Was this page helpful?
You May Also Like
Arrays in AWK
Understand AWK's associative arrays — string-keyed, dynamically growing collections used for counting, grouping, and lookups — plus multi-dimensional emulation, membership tests, and deletion.
Conditionals in AWK
Learn how AWK makes decisions using patterns, if/else statements, the ternary operator, and comparison and logical operators to selectively process input records.
User-Defined Functions in AWK
Learn to define reusable functions in AWK, understand its unusual parameter-passing rules — scalars by value, arrays by reference — and use the extra-parameter trick for local variables.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics