Building Docker Images in Pipelines
Containerized applications need their images built consistently and reproducibly as part of the CI process, turning source code into an immutable, deployable artifact. Building Docker images inside a pipeline differs from building them on a developer's laptop in several important ways: the build must run non-interactively, produce deterministic results regardless of the runner's prior state, execute quickly enough to not bottleneck the pipeline, and often needs to authenticate to a registry to push the resulting image. Pipeline-native Docker builds also introduce new failure modes — missing build context, expired registry credentials, or an image that builds successfully but fails a vulnerability scan — that pipeline authors must design around explicitly.
Cricket analogy: Building on a laptop is like a nets session where you can pause and adjust your grip, but a pipeline build is like Virat Kohli batting in a World Cup final -- no do-overs, and if his kit bag (build context) is missing or his accreditation (registry credentials) has expired, the innings fails outright.
Multi-Stage Builds for Smaller, Safer Images
A multi-stage Dockerfile uses multiple FROM statements, where an early stage installs build tools and compiles the application, and a final, much smaller stage copies only the compiled artifacts into a minimal runtime base image. This pattern is essential in CI pipelines because it keeps the final image free of compilers, package manager caches, and source code that bloat the image and expand its attack surface. A Go binary, for instance, can be compiled in a golang:1.22 stage and then copied into a distroless or alpine-based final stage that never contains the Go toolchain at all, often shrinking the final image from hundreds of megabytes to under twenty.
Cricket analogy: A multi-stage Dockerfile is like a team using specialist net bowlers only during practice and fielding a lean XI on match day -- the golang:1.22 build stage is the nets session with all the extra gear, while the alpine final stage is the stripped-down playing eleven that actually takes the field.
Layer Caching in CI Runners
Docker's layer cache can dramatically speed up rebuilds, but ephemeral CI runners typically start with an empty local Docker cache on every run, defeating this benefit unless the pipeline explicitly restores it. Modern approaches use registry-based cache backends (--cache-from and --cache-to with BuildKit) or dedicated CI cache actions that push and pull cache layers to and from a registry or blob store between runs. Ordering Dockerfile instructions from least to most frequently changing — installing dependencies before copying application source — maximizes the number of layers that can be reused across builds even without cross-run caching.
Cricket analogy: Docker's layer cache is like a bowler's rhythm built over a long spell, but a fresh CI runner is like a substitute fielder coming on cold with no rhythm built up -- teams now warm up by pulling cached 'form' from a shared video archive (registry cache), and captains bat their steadiest players (stable dependencies) first before the volatile tailenders (frequently-changing source).
name: docker-build
on:
push:
branches: [main]
jobs:
build-image:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ghcr.io/acme/api:${{ github.sha }}
cache-from: type=registry,ref=ghcr.io/acme/api:buildcache
cache-to: type=registry,ref=ghcr.io/acme/api:buildcache,mode=max
- name: Scan image for vulnerabilities
uses: aquasecurity/trivy-action@master
with:
image-ref: ghcr.io/acme/api:${{ github.sha }}
severity: HIGH,CRITICAL
exit-code: '1'BuildKit, Docker's modern build engine, executes independent Dockerfile stages and instructions in parallel and supports mounting a persistent cache directory for package managers (like apt or pip) across builds without baking that cache into the final image — a feature standard docker build never had.
A frequent mistake is copying the entire build context, including .git history, test fixtures, and local secrets files, into the Docker build by omitting a .dockerignore file. This bloats build time, increases image size, and can accidentally leak sensitive files into an image layer even if a later step deletes them, since earlier layers remain in the image history.
Build Arguments and Reproducibility
Pipelines commonly pass build arguments such as the git commit SHA, build timestamp, or version number into the image via --build-arg, embedding traceability directly into the artifact. However, any build argument that changes on every run (like a timestamp) placed early in the Dockerfile will invalidate the cache for every subsequent instruction, so such volatile arguments should be introduced as late as possible in the Dockerfile, ideally only in a final LABEL instruction, to preserve caching for the expensive steps above it.
Cricket analogy: Stamping a Docker image with a build timestamp is like writing today's date on a bat before an innings -- useful for traceability, but if you engrave it early in the manufacturing process it forces the whole bat to be remade, so factories add the date sticker (LABEL) only at the very last finishing step.
- Multi-stage builds separate compile-time tooling from the runtime image, reducing size and attack surface.
- CI runners start with cold Docker caches, so registry-based cache-from/cache-to is needed to reuse layers across runs.
- Ordering Dockerfile instructions from least to most frequently changing maximizes cache reuse.
- A .dockerignore file prevents bloated builds and accidental leakage of sensitive files into image layers.
- Volatile build arguments (timestamps, SHAs) should be introduced late in the Dockerfile to avoid busting earlier cache layers.
- Vulnerability scanning should run immediately after the build, before the image is promoted or deployed.
Practice what you learned
1. What is the main benefit of a multi-stage Dockerfile in a CI pipeline?
2. Why do ephemeral CI runners typically fail to benefit from Docker's local layer cache without extra configuration?
3. Why should volatile build arguments like a timestamp be placed late in the Dockerfile?
4. What risk does omitting a .dockerignore file introduce?
5. Where should an image vulnerability scan typically run in a build pipeline?
Was this page helpful?
You May Also Like
Container Registries and Tagging Strategies
Explore how CI/CD pipelines push images to container registries and the tagging conventions that keep deployments traceable, safe, and repeatable.
Deploying Containers from CI/CD
Learn the common patterns pipelines use to deploy containerized applications, from direct kubectl updates to GitOps-driven reconciliation.
Caching and Build Artifacts
Learn how CI/CD pipelines speed up builds with dependency caching and pass data between jobs using artifacts, and where each technique fits.
Pipeline Security Basics
CI/CD pipelines are a high-value attack surface with broad access to source, secrets, and production, so securing them requires trusted inputs, isolated execution, and verified artifacts.
Related Reading
Related Study Notes in DevOps
Browse all study notesNginx Study Notes
DevOps · 30 topics
DevOpsAnsible Study Notes
DevOps · 30 topics
DevOpsAdvanced Kubernetes Study Notes
Kubernetes · 30 topics
DevOpsAdvanced Bash Scripting Study Notes
Bash · 30 topics
DevOpsApache Kafka Study Notes
Kafka · 30 topics
DevOpsDocker & Kubernetes Study Notes
YAML · 40 topics