What is Log Rotation and Why is it Necessary?
Learn what log rotation is, how logrotate works with compress and retention, and why containers rotate logs differently.
Expected Interview Answer
Log rotation is the automated process of archiving, compressing, and eventually deleting old log files on a schedule or size threshold, so a continuously growing log file never fills the disk and older logs remain available for a bounded retention window instead of accumulating forever.
A tool like logrotate is typically driven by cron and runs a policy per log file: rotate daily or once the file exceeds a size like 100MB, rename the current file with a suffix or timestamp, compress the rotated file with gzip to save space, and delete rotated files older than a configured retention count or age. Applications must be signaled (commonly via SIGHUP or a copytruncate strategy) after rotation so they reopen a fresh file handle instead of continuing to write into a now-renamed, unlinked file — a mistake that silently loses new log data. In containerized environments, rotation is often handled instead by the container runtime’s log driver (with max-size and max-file options) or by the log shipper itself, since containers are ephemeral and local rotation matters less once logs are streamed to centralized storage. Without rotation, an unbounded log file can fill the root or data disk, which on many systems causes cascading failures well beyond just losing logs.
- Prevents disk exhaustion from unbounded log growth
- Keeps a bounded, predictable retention window for compliance/debugging
- Reduces storage cost via compression of older logs
- Avoids performance degradation from writing to a single huge file
AI Mentor Explanation
Log rotation is like a scorer archiving each match’s full scorebook at the end of the day instead of writing every match of the season into one endless notebook. The current day’s book stays open and active for writing, while yesterday’s finished book gets filed away, and books older than the league’s retention policy get discarded to make room. If the scorer kept appending forever into one notebook, it would eventually become too bulky to carry or search through quickly. Archiving daily keeps each book a manageable, findable size while still preserving recent match history.
Step-by-Step Explanation
Step 1
Define a rotation policy
Configure rotation trigger (daily, weekly, or a size threshold like 100MB) and retention count per log file.
Step 2
Rotate the file
The old log is renamed or timestamped, and a new empty file is created for continued writes.
Step 3
Signal the application
Send SIGHUP or use copytruncate so the app reopens a fresh file handle instead of writing into an orphaned inode.
Step 4
Compress and expire
Rotated files are gzip-compressed, and files beyond the retention count/age are deleted automatically.
What Interviewer Expects
- Understanding that unbounded logs can exhaust disk space
- Knowledge of logrotate mechanics: rotate, compress, retain, signal
- Awareness of the SIGHUP/copytruncate problem for open file handles
- Understanding of how rotation differs in containerized environments
Common Mistakes
- Forgetting to signal the app after rotation, silently losing new log writes
- Not setting a retention limit, letting compressed archives accumulate forever
- Assuming container logs need the same local rotation as VM-based logs
- Rotating based only on time and ignoring a size-based safety threshold
Best Answer (HR Friendly)
“Log rotation automatically archives and compresses old log files on a schedule, so our disks never fill up from logs that would otherwise grow forever. It keeps a reasonable window of recent logs available for debugging while making sure disk space and performance stay predictable, without anyone having to manually clean up log files.”
Code Example
/var/log/myapp/*.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
size 100M
postrotate
kill -HUP $(cat /var/run/myapp.pid) 2>/dev/null || true
endscript
}Follow-up Questions
- What is the difference between SIGHUP-based rotation and copytruncate?
- How does log rotation work differently for containerized applications?
- How would you set a retention policy that satisfies a compliance requirement?
- What happens if an application keeps writing to a rotated, unlinked file?
MCQ Practice
1. What is the primary risk of not rotating logs?
Without rotation, a continuously growing log file can consume all available disk space, potentially causing broader system failures.
2. Why must an application be signaled (e.g. SIGHUP) after log rotation?
Without reopening the file handle, the app keeps writing into the old inode even after the file was renamed, silently losing new log data from the new file.
3. In containerized environments, where is log rotation commonly handled instead of locally?
Since containers are ephemeral, rotation is often delegated to the runtime’s log driver (max-size/max-file) or handled once logs are shipped to centralized storage.
Flash Cards
What is log rotation? — Automated archiving, compressing, and deleting of old log files on a schedule or size threshold.
Why is it necessary? — To prevent unbounded log growth from exhausting disk space.
Why signal the app after rotation? — So it reopens a fresh file handle instead of writing into a renamed, unlinked file.
How does container log rotation differ? — Often handled by the container runtime log driver or the centralized log shipper instead of local logrotate.