Reading Multiple Files
AWK accepts any number of input files on the command line and processes them sequentially, as though they were concatenated into one stream. What makes it powerful is that AWK exposes variables telling you exactly which file and which line you are on at every moment. This makes AWK a natural tool for merging logs, joining datasets, and generating per-file summaries in a single pass.
Cricket analogy: Like scoring a Test series across several match scorecards as one running series tally, while still knowing which match and which innings each delivery belongs to.
FILENAME, FNR, and NR
Three built-in variables track your position. NR, the Number of Records, counts every record from the very first file onward and never resets. FNR, the File Number of Records, resets to 1 whenever a new file begins. FILENAME holds the name of the file currently being read; it is empty inside a BEGIN block because no file has been opened yet. Together they let you distinguish a record's global position from its position within its own file.
Cricket analogy: Like a series aggregate where NR is the running total of balls faced across all Tests, FNR is the ball count within the current innings, and FILENAME is which stadium you are at.
# Label each record with its file, its line-in-file, and its global line
{ printf "%s (file line %d, total line %d): %s\n", FILENAME, FNR, NR, $0 }
# Run: awk -f label.awk access-jan.log access-feb.logDetecting File Boundaries with FNR==1
Because FNR resets to 1 at the top of each file, the condition FNR==1 fires exactly once per file. That makes it a clean hook for printing a per-file header, resetting per-file accumulators, or capturing a file's header row before processing its data. Pairing FNR==1 with FILENAME lets you label each file's section in a combined report without any external bookkeeping.
Cricket analogy: Like the fresh coin toss and team sheet read out at the start of every match, FNR==1 is the opening ceremony that happens once per game.
# Print a labelled header before each file's contents
FNR == 1 { printf "\n===== %s =====\n", FILENAME }
{ print }The nextfile statement immediately stops processing the current file and jumps to the next one. Long a gawk extension, it was standardized by POSIX in 2008. It is ideal when you only need the first few lines of each file — for example, checking every file's header without reading the whole thing.
Correlating Data Across Files
The idiom NR==FNR is true only while the first file is being read, because only there do NR and FNR advance in lockstep; once the second file starts, NR keeps climbing while FNR restarts, so NR is greater than FNR. Programmers exploit this to load a lookup table from the first file into an associative array, then use it while streaming the second file — the classic AWK join. The block usually ends with next so the first file's records skip the second rule.
Cricket analogy: Like memorizing the opposition's batting order from the team sheet before play, then looking up each batsman as they walk out to the crease.
# Join users.txt (id name) with orders.txt (id amount) on the id
NR == FNR { name[$1] = $2; next } # first file: build the lookup
$1 in name { printf "%s spent %s\n", name[$1], $2 } # second file: use itThe NR==FNR trick assumes the first file is non-empty. If it is empty, NR and FNR stay in lockstep into the second file, so your 'first file' block runs on the wrong data and the join silently breaks. When files may be empty, guard the logic by comparing FILENAME against the expected first filename instead.
- AWK processes multiple input files sequentially as one continuous stream.
- NR counts records across all files and never resets; FNR resets to 1 at each new file.
- FILENAME holds the name of the file currently being read.
- FNR==1 fires once per file, perfect for per-file headers or accumulator resets.
- NR==FNR is true only while the first file is read, the basis of the AWK join idiom.
- next skips the remaining rules for a record; nextfile skips the rest of a file.
- An empty first file breaks the NR==FNR idiom; guard with FILENAME if needed.
Practice what you learned
1. Which variable resets to 1 at the start of each new input file?
2. When is the condition NR==FNR true?
3. What does the variable FILENAME contain?
4. What is a common use of the FNR==1 pattern?
5. What does the nextfile statement do?
Was this page helpful?
You May Also Like
Built-in Variables in AWK
AWK ships with a set of predefined variables such as NR, NF, FS, and OFS that expose the current record, field counts, and formatting controls without any setup.
Arrays in AWK
Understand AWK's associative arrays — string-keyed, dynamically growing collections used for counting, grouping, and lookups — plus multi-dimensional emulation, membership tests, and deletion.
AWK in Shell Pipelines
How to slot AWK into Unix pipelines as a field-aware filter and transformer, passing data between grep, sort, cut, and other tools via stdin and stdout.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics