What is Infrastructure Drift and How Do You Detect and Prevent It?
Learn what infrastructure drift is, how terraform plan detects it, and how IAM restrictions prevent it — with a real DevOps interview answer.
Expected Interview Answer
Infrastructure drift happens when the real state of provisioned infrastructure diverges from what its infrastructure-as-code definition declares — usually because someone made a manual change directly in the console — and it is detected by regularly running a plan/diff comparison against the live state and prevented by locking down manual access and enforcing that all changes flow through code.
Drift typically starts small: an engineer tweaks a security group rule directly in the AWS console to unblock an incident, and forgets to reflect that change back into the Terraform configuration. Terraform stores its last-known state and, when you run `terraform plan`, it queries the real cloud provider APIs and diffs the actual resource state against both the state file and the declared configuration, surfacing any drift as an unexpected planned change. Left undetected, drift compounds — the next `terraform apply` might silently revert someone’s emergency fix, or worse, the team loses confidence in IaC entirely because “the code doesn’t match reality” and reverts to manual changes, defeating the purpose of IaC. The fix is process plus tooling: run scheduled drift-detection plans in CI, restrict console write access via IAM policies so manual changes are hard to make in the first place, and require any legitimate manual emergency change to be immediately backported into the IaC source and applied through the pipeline.
- Keeps the IaC source of truth actually trustworthy
- Prevents emergency fixes from being silently reverted by the next apply
- Surfaces unauthorized or undocumented changes for audit purposes
- Preserves reproducibility — infra can be rebuilt confidently from code
AI Mentor Explanation
Infrastructure drift is like a groundstaff member watering one specific patch of the pitch differently from the approved schedule to fix a dry spot, without updating the maintenance log. The official log still says standard watering was applied, but the actual pitch now behaves differently in that one area. If nobody notices, the next scheduled maintenance crew follows the outdated log and creates an inconsistency with the pitch’s real, drifted condition. Regular pitch inspections that compare the log against the actual surface catch this drift before it affects a match.
Step-by-Step Explanation
Step 1
Run scheduled drift detection
Run `terraform plan` (or equivalent) on a schedule in CI to diff live cloud state against the declared config.
Step 2
Alert on unexpected diffs
Fail the pipeline or notify the team when a plan shows changes nobody intentionally authored via code.
Step 3
Restrict manual write access
Lock down console/CLI write permissions via IAM so infrastructure changes are hard to make outside the pipeline.
Step 4
Backport emergency changes
Require any legitimate manual fix to be immediately reflected back into the IaC source and reapplied through the pipeline.
What Interviewer Expects
- Clear definition of drift as a mismatch between declared IaC and actual live state
- Understanding of how terraform plan detects drift by querying real provider APIs
- Awareness of both detection tooling and preventative process (IAM restrictions)
- Recognition of the risk that the next apply can silently revert an undocumented manual fix
Common Mistakes
- Believing IaC alone prevents drift without also restricting manual console access
- Never running scheduled drift-detection plans, only checking at apply time
- Manually fixing an incident without backporting the change into the IaC source
- Ignoring drift warnings because “the plan usually shows noise”
Best Answer (HR Friendly)
“Infrastructure drift happens when someone changes a resource directly in the cloud console instead of through our Terraform code, so the code no longer matches reality. We catch it by running scheduled plan checks that compare our code against the live infrastructure and flag any unexpected differences. To prevent it in the first place, we restrict who can make manual changes and make sure any true emergency fix gets reflected back into the code right away.”
Code Example
# Run in CI on a schedule (e.g. nightly) to detect drift
terraform init -input=false
terraform plan -detailed-exitcode -out=drift.tfplan
# Exit codes: 0 = no changes, 1 = error, 2 = drift detected
if [ $? -eq 2 ]; then
echo "Drift detected: live infrastructure differs from IaC"
exit 1
fiFollow-up Questions
- How would you handle a legitimate emergency manual change without causing future drift?
- What IAM restrictions would you put in place to reduce console-level drift?
- How does Terraform state locking relate to preventing drift from concurrent applies?
- What is the risk of running terraform apply immediately after detecting drift, without review?
MCQ Practice
1. What is infrastructure drift?
Drift specifically refers to the real infrastructure state diverging from what the IaC configuration declares, usually due to manual out-of-band changes.
2. What command is commonly used to detect Terraform drift?
`terraform plan` queries the real provider APIs and diffs actual resource state against the state file and declared configuration, surfacing drift.
3. What is a key preventative measure against infrastructure drift?
Restricting manual write access makes it harder to introduce out-of-band changes that cause the live state to diverge from the IaC source.
Flash Cards
What is infrastructure drift? — A divergence between the live infrastructure state and its declared IaC configuration.
How is drift detected? — Running terraform plan (or equivalent) to diff real cloud state against the declared config and state file.
Main cause of drift? — Manual out-of-band changes made directly in the console instead of through code.
Key prevention measure? — Restrict manual write access via IAM and require all changes to flow through the pipeline.