What is the Prometheus scrape configuration and how do targets work?
Understand Prometheus scrape configuration: how jobs, targets, scrape_interval, job and instance labels, and the up metric work to pull metrics from endpoints.
Expected Interview Answer
A scrape configuration is a job definition in prometheus.yml that tells Prometheus which HTTP endpoints (targets) to pull metrics from, how often, and how to label the resulting series.
Each entry under scrape_configs defines a job_name and a set of targets, supplied either statically via static_configs or dynamically via service discovery. Prometheus periodically issues an HTTP GET to each target's /metrics path at the configured scrape_interval, honoring scrape_timeout, and stores every returned sample tagged with a job label and an instance label. Targets can be filtered and relabeled, and each scrape's success is exposed through the built-in up metric.
- Centralizes what to monitor and how often in one file
- Supports both static targets and dynamic service discovery
- Automatic job and instance labels identify every series
- Per-job scrape_interval and scrape_timeout tuning
- The up metric makes target health observable
AI Mentor Explanation
A scrape config is like a coach's fixed routine of walking to each net at set intervals to jot down every batter's runs and dismissals. The list of nets is the targets, the walking rhythm is the scrape_interval, and the notebook column he writes under is the job label. If a net is empty when he arrives, he marks it absent, exactly as Prometheus records up as zero when a target fails to answer its scrape.
Step-by-Step Explanation
Step 1
Define a job
Add an entry under scrape_configs with a unique job_name; this value becomes the job label on every scraped series.
Step 2
Provide targets
List endpoints in static_configs or attach a service discovery block so Prometheus knows what to scrape.
Step 3
Set the endpoint details
Optionally override metrics_path (default /metrics), scheme (http/https), and any params or basic_auth needed.
Step 4
Tune timing
Set scrape_interval and scrape_timeout at the global or per-job level to control frequency and cutoff.
Step 5
Scrape and label
Prometheus issues periodic GETs, tagging samples with job and instance labels and recording the up metric per target.
What Interviewer Expects
- Knowledge that scrape_configs defines jobs and their targets
- Understanding of static_configs vs service discovery for targets
- Awareness of the default /metrics path and scrape_interval
- The meaning of the automatic job and instance labels
- How the up metric reflects scrape success or failure
Common Mistakes
- Thinking Prometheus pushes to targets rather than pulling from them
- Forgetting the default metrics_path is /metrics
- Confusing scrape_interval with evaluation_interval for rules
- Not knowing job and instance labels are added automatically
- Ignoring scrape_timeout when a target is slow to respond
Best Answer (HR Friendly)
“A scrape configuration is the part of Prometheus that lists which servers to collect metrics from and how often. Prometheus regularly visits each server's metrics page, records the numbers, and labels them so you can tell which machine and job they came from.”
Code Example
global:
scrape_interval: 15s
scrape_timeout: 10s
scrape_configs:
- job_name: 'api-service'
metrics_path: /metrics
scheme: http
scrape_interval: 30s # overrides the global value for this job
static_configs:
- targets:
- '10.0.0.11:9100'
- '10.0.0.12:9100'
labels:
env: productionFollow-up Questions
- What is the difference between scrape_interval and evaluation_interval?
- What does the up metric tell you about a target?
- How do you scrape an endpoint that requires authentication?
- What is the default metrics_path and how do you change it?
- How do static_configs and service discovery differ for supplying targets?
MCQ Practice
1. What is the default HTTP path Prometheus scrapes on each target?
Unless metrics_path is overridden, Prometheus issues its scrape GET request to the /metrics path on each target.
2. Which two labels does Prometheus automatically attach to scraped series?
Every scraped sample gets a job label from the job_name and an instance label identifying the target endpoint.
3. What does a value of 0 for the up metric indicate?
Prometheus records up as 1 for a successful scrape and 0 when the target is unreachable or times out.
Flash Cards
Where are scrape targets defined? — Under scrape_configs in prometheus.yml, via static_configs or a service discovery block.
Default metrics_path? — /metrics, unless overridden per job.
Which labels are auto-added? — job (from job_name) and instance (the target endpoint).
What is the up metric? — A per-target metric set to 1 on a successful scrape and 0 on failure or timeout.
Does Prometheus push or pull? — It pulls, issuing periodic HTTP GETs to each target's metrics endpoint.