Building Docker Images in Pipelines

Containerized applications need their images built consistently and reproducibly as part of the CI process, turning source code into an immutable, deployable artifact. Building Docker images inside a pipeline differs from building them on a developer's laptop in several important ways: the build must run non-interactively, produce deterministic results regardless of the runner's prior state, execute quickly enough to not bottleneck the pipeline, and often needs to authenticate to a registry to push the resulting image. Pipeline-native Docker builds also introduce new failure modes — missing build context, expired registry credentials, or an image that builds successfully but fails a vulnerability scan — that pipeline authors must design around explicitly.

A multi-stage Dockerfile uses multiple FROM statements, where an early stage installs build tools and compiles the application, and a final, much smaller stage copies only the compiled artifacts into a minimal runtime base image. This pattern is essential in CI pipelines because it keeps the final image free of compilers, package manager caches, and source code that bloat the image and expand its attack surface. A Go binary, for instance, can be compiled in a golang:1.22 stage and then copied into a distroless or alpine-based final stage that never contains the Go toolchain at all, often shrinking the final image from hundreds of megabytes to under twenty.

Docker's layer cache can dramatically speed up rebuilds, but ephemeral CI runners typically start with an empty local Docker cache on every run, defeating this benefit unless the pipeline explicitly restores it. Modern approaches use registry-based cache backends (--cache-from and --cache-to with BuildKit) or dedicated CI cache actions that push and pull cache layers to and from a registry or blob store between runs. Ordering Dockerfile instructions from least to most frequently changing — installing dependencies before copying application source — maximizes the number of layers that can be reused across builds even without cross-run caching.

Pipelines commonly pass build arguments such as the git commit SHA, build timestamp, or version number into the image via --build-arg, embedding traceability directly into the artifact. However, any build argument that changes on every run (like a timestamp) placed early in the Dockerfile will invalidate the cache for every subsequent instruction, so such volatile arguments should be introduced as late as possible in the Dockerfile, ideally only in a final LABEL instruction, to preserve caching for the expensive steps above it.