CI/CD Pipeline Design Cheat Sheet
Core principles and patterns for structuring continuous integration and delivery pipelines, from stages to caching and artifacts.
Typical Pipeline Stages
A common progression from commit to production.
- Lint / Static Analysis- Fast checks (linters, type checkers) that fail cheaply before expensive stages run
- Build- Compile code and produce a versioned, immutable artifact (binary, image, package)
- Unit Tests- Fast, isolated tests run against the build on every commit
- Integration/E2E Tests- Slower tests against real dependencies, often run on a subset of branches
- Security Scan- SAST/dependency scanning (e.g. Trivy, Snyk) gating merges on critical findings
- Deploy to Staging- Automated deploy of the same artifact to a production-like environment
- Deploy to Production- Promotion of the exact tested artifact, often behind a manual approval gate
Example: GitHub Actions Pipeline
A concise CI workflow with caching and a build artifact.
name: CIon: [push, pull_request]jobs: build-test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 cache: 'npm' - run: npm ci - run: npm run lint - run: npm test -- --ci - run: npm run build - uses: actions/upload-artifact@v4 with: name: dist path: dist/
Design Principles
What separates a reliable pipeline from a flaky one.
- Build once, promote many- Build a single artifact and promote it unchanged through each environment, never rebuild per stage
- Fail fast- Order cheap/fast checks (lint, unit tests) before slow ones (E2E, security scans)
- Idempotent deploys- Re-running a deploy step should produce the same end state, not duplicate side effects
- Immutable artifacts- Tag builds with a commit SHA or version, never overwrite 'latest' as the deploy source of truth
- Parallelization- Run independent jobs (lint, unit tests, security scan) concurrently to shorten pipeline time
- Environment parity- Keep staging as close to production as feasible to catch environment-specific bugs early
Matrix Builds & Dependency Caching
Fan out across versions/platforms while reusing cached dependencies between runs.
jobs: test: strategy: fail-fast: false matrix: node: [18, 20, 22] os: [ubuntu-latest, macos-latest] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: ${{ matrix.node }} - uses: actions/cache@v4 with: path: ~/.npm key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }} restore-keys: npm-${{ runner.os }}- - run: npm ci - run: npm test
Canary Rollout Gate
Shift traffic incrementally and auto-rollback on error-rate regressions.
deploy-canary: needs: build steps: - run: kubectl argo rollouts set image app app=$IMAGE:$SHA - run: | argo rollouts promote app --step 1 # shift 10% traffic - name: Watch error rate run: | for i in $(seq 1 10); do rate=$(curl -s $METRICS_URL/error_rate) if (( $(echo "$rate > 0.02" | bc -l) )); then argo rollouts abort app exit 1 fi sleep 30 done - run: argo rollouts promote app --full
Common Pipeline Anti-Patterns
Failure modes that quietly erode trust in a CI/CD system.
- Snowflake pipelines- Each service's pipeline is hand-edited and diverges, making shared fixes impossible; solve with reusable workflows/templates
- Flaky test masking- Auto-retrying failed tests without quarantining them hides real regressions and erodes trust in red/green signal
- Secrets baked into images- Injecting credentials at build time instead of runtime leaks them into every layer and every consumer of the artifact
- Unbounded pipeline duration- No timeout on jobs means a hung step blocks the queue indefinitely instead of failing fast and freeing runners
- Manual approval theater- A human 'approve' click with no real verification step attached provides false confidence, not a safety gate
- Shared mutable staging- One staging environment serving many concurrent PRs causes cross-contamination; prefer ephemeral per-PR environments
Artifact Signing & Provenance (SLSA)
Sign and attest build artifacts so downstream consumers can verify origin and integrity.
sign-and-attest: needs: build permissions: id-token: write # for keyless OIDC signing contents: read attestations: write steps: - uses: actions/checkout@v4 - run: cosign sign --yes $IMAGE@$DIGEST - uses: actions/attest-build-provenance@v1 with: subject-name: ${{ env.IMAGE }} subject-digest: ${{ env.DIGEST }} - run: cosign verify --certificate-identity-regexp '.*' --certificate-oidc-issuer https://token.actions.githubusercontent.com $IMAGE@$DIGEST
Decoupling Deploy from Release
Ship code dark behind a flag so deployment and feature exposure become independent events.
deploy: steps: - run: kubectl set image deployment/app app=$IMAGE:$SHA - run: kubectl rollout status deployment/app --timeout=120srelease: needs: deploy environment: production steps: - name: Enable flag for 5% of users run: | curl -X PATCH $FLAGS_API/flags/new-checkout \ -d '{"rollout": 5, "enabled": true}' - name: Monitor then ramp run: ./scripts/ramp-flag.sh new-checkout
Build your deployable artifact exactly once per commit and pass that same artifact through every downstream stage — rebuilding at each stage risks deploying code that was never actually tested.