Observability in Microservices

Observability is the ability to ask arbitrary questions about a running system's internal state using only the external signals it emits, without shipping new code to answer each new question. In a monolith a single stack trace often tells you everything; in a microservices architecture a single user request can hop across a dozen services, so you need logs, metrics, and traces working together to reconstruct what actually happened.

Logs are discrete, timestamped events emitted by a service (a request was received, an exception was thrown); metrics are numeric time-series aggregates (request rate, error count, p99 latency) that are cheap to store and great for dashboards and alerting; traces capture the end-to-end path and timing of a single request as it crosses service boundaries. Each pillar answers a different question: logs answer 'what happened,' metrics answer 'how much/how often,' and traces answer 'where did the time go.'

Structured logging means emitting logs as machine-parseable key-value pairs (usually JSON) instead of free-text sentences, so log aggregators can index and query fields like service, order_id, or error_code directly. The critical field for microservices is a correlation ID (often the same value as the trace ID) generated at the system's edge and propagated on every downstream call via an HTTP header or message metadata, so every log line touched by one user request can be joined together even though it was written by five different services on five different hosts.

Prometheus scrapes numeric metrics from each service's /metrics endpoint on a fixed interval and stores them as time series, which Grafana then visualizes and Alertmanager uses to fire alerts when thresholds are breached. The RED method (Rate, Errors, Duration) is the standard set of metrics to expose for every request-driven microservice, while the USE method (Utilization, Saturation, Errors) is better suited to resources like CPU, memory, or connection pools.