Why Run Selenium in CI
Running Selenium tests only on developer laptops means UI regressions are caught late, inconsistently, or not at all before code merges. Wiring a Selenium suite into a CI pipeline — GitHub Actions, GitLab CI, Jenkins, or CircleCI — means every pull request automatically spins up a browser environment, runs the suite headlessly against a freshly deployed build or preview environment, and blocks the merge if UI tests fail, turning Selenium from a manual, occasional check into an enforced quality gate that runs identically for every contributor regardless of their local machine setup.
Cricket analogy: It's like the BCCI mandating a standardized fitness Yo-Yo test before every player is selected, rather than trusting each player's self-reported fitness, so no one gets picked without passing the same objective check.
A Typical CI Job Structure
A typical Selenium CI job installs the application dependencies, starts the application under test (either locally via docker-compose up or against a deployed preview URL), installs browser binaries via Selenium Manager or a base Docker image like selenium/standalone-chrome, runs the test suite headlessly with parallelism, and uploads failure artifacts — screenshots, HTML reports, and video recordings if using a Grid provider — before reporting pass/fail status back to the pull request via a status check. Caching the browser binaries and dependency installation step between runs meaningfully cuts pipeline time, since re-downloading Chrome or re-installing pip/npm packages on every run adds unnecessary minutes to every single commit.
Cricket analogy: It's like a match day sequence — pitch inspection, toss, warm-up, the actual match, then post-match report — each step happening in a fixed, repeatable order every single game.
# .github/workflows/selenium-tests.yml
name: Selenium UI Tests
on: [pull_request]
jobs:
ui-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install dependencies
run: pip install -r requirements.txt
- name: Start application under test
run: docker compose up -d --wait
- name: Run Selenium suite (headless, parallel)
run: pytest -n 4 tests/ui --dist=loadscope --headless
- name: Upload failure artifacts
if: failure()
uses: actions/upload-artifact@v4
with:
name: selenium-failure-artifacts
path: artifacts/Environments, Flakiness, and Retries
CI Selenium suites should target a freshly provisioned environment per run — either an ephemeral preview deployment or a docker-compose stack seeded with known test data — rather than a shared staging environment that other test runs or manual QA might mutate concurrently, since that shared-state problem is what causes the majority of 'flaky in CI, fine locally' reports. A pragmatic retry policy (retrying a failed test once or twice before marking the build red, via pytest-rerunfailures or a CI-native retry setting) is reasonable for genuinely environmental flakiness like a slow third-party API, but retries should never be used to paper over a real, reproducible bug — a test failing consistently across all retries is signal, not noise.
Cricket analogy: It's like insisting every team practices on a freshly rolled, dedicated pitch rather than a shared ground other teams are also training on simultaneously, since a shared pitch getting churned up mid-session is what causes unpredictable bounce.
Track flaky-test rate as an explicit metric over time (many CI dashboards and tools like pytest's --reruns reporting support this) — a rising flaky rate is an early warning sign of environment instability or race conditions long before it becomes a full pipeline outage.
Never silently swallow a Selenium test failure by wrapping assertions in a broad try/except that only logs a warning — this defeats the entire purpose of the CI gate and lets real regressions merge to main undetected while the pipeline shows a comforting but false green checkmark.
- CI integration turns Selenium from a manual, occasional check into an enforced quality gate on every pull request.
- A typical job installs dependencies, starts the app under test, runs headless parallel tests, and uploads failure artifacts.
- Caching browser binaries and package installs meaningfully reduces per-run pipeline time.
- Tests should target a freshly provisioned or ephemeral environment, not a shared staging environment other processes can mutate.
- A limited, explicit retry policy is reasonable for genuine environmental flakiness but should never mask a reproducible bug.
- Tracking flaky-test rate over time surfaces environment instability before it causes a full pipeline outage.
- Never silently catch and log test failures — that defeats the purpose of the CI gate entirely.
Practice what you learned
1. What is the main benefit of running Selenium tests in a CI pipeline rather than only on developer laptops?
2. Why should CI Selenium suites target a freshly provisioned or ephemeral environment rather than shared staging?
3. What is an appropriate use of test retries in a CI pipeline?
4. What should be uploaded as CI artifacts when a Selenium test fails?
5. Why is silently catching and logging a Selenium assertion failure in a broad try/except considered harmful in CI?
Was this page helpful?
You May Also Like
Headless Browser Testing
Run Selenium tests against browsers with no visible UI, speeding up CI execution while understanding the tradeoffs versus headed mode.
Parallel Test Execution
Speed up Selenium suites by running multiple tests concurrently across threads, processes, or Grid nodes without sessions interfering with each other.
Selenium Grid Basics
Learn how Selenium Grid distributes WebDriver sessions across multiple machines and browsers so test suites can run remotely and at scale.