The Union Filesystem and Layers
Each instruction in a Dockerfile that modifies the filesystem (RUN, COPY, ADD) produces a new, immutable layer, identified by a content hash. Docker stacks these layers using a union filesystem (commonly overlay2 on Linux) to present them as a single unified filesystem to the container, while storing them separately and sharing common layers across images.
Cricket analogy: Each stage of a match report - toss result, innings score, fall of wickets - is recorded as a distinct, unchangeable entry with its own reference number, and the full scorecard stacks these entries together, just as Docker layers stack via a union filesystem while stored separately.
How the Build Cache Works
During docker build, Docker evaluates each instruction in order. For each one, it checks whether a layer already exists in the cache that was produced by the exact same instruction with the same preceding layer chain. If so, it reuses that cached layer instead of re-executing the instruction. As soon as one instruction misses the cache, every subsequent instruction in the Dockerfile also misses, even if it hasn't changed — cache invalidation cascades downward.
Cricket analogy: When reviewing a rain-affected match, the umpires check if the exact same sequence of overs and conditions occurred before reusing a prior Duckworth-Lewis calculation; once one over's conditions differ, every subsequent recalculation must be redone too, mirroring cascading cache invalidation.
For COPY and ADD, Docker checks the actual checksum of the files being copied, not just the instruction text, to decide on a cache hit. For RUN, Docker compares the literal command string.
Ordering Instructions for Maximum Cache Reuse
Place instructions that change infrequently (installing OS packages, copying dependency manifests, installing dependencies) before instructions that change frequently (copying application source code). This way, editing a single source file only invalidates the cache from that COPY instruction onward, not the expensive dependency-install step above it.
Cricket analogy: A team sets its base training routine (fitness, fielding drills) early in preseason since it rarely changes, and only finalizes the specific batting order (which changes every match) last, so a single lineup swap doesn't force redoing the whole preseason fitness program.
# Poor ordering — invalidates dependency install on every source change
FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "app.py"]
# --- vs ---
# Good ordering — dependency layer only rebuilds when requirements.txt changes
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "app.py"]Reducing Layer Count and Size
Every RUN instruction adds a layer, and files deleted in a later layer still consume space in earlier layers because layers are additive diffs, not merges. Chaining related commands into a single RUN with '&&' and cleaning up temporary files within that same instruction avoids leaving stale data behind in the image.
Cricket analogy: If a groundskeeper spreads extra sand on the pitch during preparation and later 'removes' it by raking over it, the sand is still physically underneath; similarly, files deleted in a later Docker layer still consume space in earlier layers because layers are additive diffs.
# Bad: apt cache remains in an earlier layer even though it's 'removed' later
RUN apt-get update
RUN apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*
# Good: single layer, cache cleaned up before the layer is committed
RUN apt-get update && \
apt-get install -y --no-install-recommends curl && \
rm -rf /var/lib/apt/lists/*Splitting cleanup into a separate RUN instruction does not shrink the image — the deleted files were already committed to an earlier, immutable layer. Only combining install-and-cleanup into one RUN (or using multi-stage builds) actually removes that weight from the final image.
Inspecting Cache Behavior
During a build, Docker's output indicates 'CACHED' next to steps that reused an existing layer. You can force a full rebuild ignoring the cache with --no-cache, which is useful when debugging suspicious caching issues or ensuring you get fully updated base image content.
Cricket analogy: During a replay review, the broadcast graphic flags 'ARCHIVE' next to footage reused from an earlier over instead of a fresh camera feed, but the director can force a full fresh re-shoot ignoring all archive footage if something looks suspicious, mirroring 'CACHED' output and '--no-cache'.
docker build -t myapp:1.0 .
# Step 3/7 : COPY requirements.txt .
# ---> Using cache
# ---> 4f2e1a9b8c3d
# Step 4/7 : RUN pip install -r requirements.txt
# ---> Using cache
# ---> 9a8b7c6d5e4f
# Force a clean rebuild, ignoring all cached layers
docker build --no-cache -t myapp:1.0 .- Each RUN, COPY, and ADD instruction produces a new immutable layer.
- Cache invalidation cascades: once one instruction misses cache, all following instructions rebuild too.
- Order Dockerfile instructions from least-frequently-changing to most-frequently-changing.
- Files deleted in a later layer still occupy space in the image because layers store diffs, not merged states.
- Combine install-and-cleanup steps into a single RUN to actually reduce image size.
- Use
docker build --no-cacheto force a full rebuild when cache correctness is in doubt.
Practice what you learned
1. What happens to the Docker build cache once a single instruction fails to match a cached layer?
2. Why should COPY requirements.txt and RUN pip install typically appear before COPY . . in a Dockerfile?
3. If you install a package with apt-get in one RUN instruction and delete its cache files in a separate, later RUN instruction, what happens to the final image size?
4. What does `docker build --no-cache` do?
5. For a COPY instruction, what does Docker check to decide whether the cache can be reused?
Was this page helpful?
You May Also Like
Writing a Dockerfile
A practical guide to Dockerfile syntax and the core instructions used to build a custom image from scratch.
Multi-Stage Builds
How to use multiple FROM stages in a single Dockerfile to separate build-time tooling from the final runtime image, producing smaller, more secure images.
Image Size Optimization
Techniques for shrinking Docker images through multi-stage builds, minimal base images, and layer-aware Dockerfile authoring to speed up builds and deployments.
Docker Images Explained
An introduction to what Docker images are, how they differ from containers, and how their layered, read-only filesystem structure works.
Related Reading
Related Study Notes in DevOps
Browse all study notesNginx Study Notes
DevOps · 30 topics
DevOpsAnsible Study Notes
DevOps · 30 topics
DevOpsAdvanced Kubernetes Study Notes
Kubernetes · 30 topics
DevOpsAdvanced Bash Scripting Study Notes
Bash · 30 topics
DevOpsApache Kafka Study Notes
Kafka · 30 topics
DevOpsCI/CD Tools & Pipelines Study Notes
YAML · 37 topics