Cloud Monitoring & Logging Cheat Sheet
Covers metrics, logs, traces, alerting, and the tooling used to observe cloud workloads across AWS CloudWatch and open standards.
2 PagesIntermediateFeb 5, 2026
Three Pillars of Observability
The core data types used to understand system behavior.
- Metrics- Numeric time-series data (CPU%, request count, latency) aggregated over time
- Logs- Discrete timestamped event records, structured or unstructured
- Traces- End-to-end records of a request's path across distributed services, with spans
- Alerting- Automated notification triggered when a metric crosses a defined threshold
- Dashboards- Visual aggregation of metrics/logs for at-a-glance system health
Create a CloudWatch Alarm
Trigger SNS notification when average CPU exceeds 80% for 3 periods.
bash
aws cloudwatch put-metric-alarm \ --alarm-name high-cpu \ --metric-name CPUUtilization \ --namespace AWS/EC2 \ --statistic Average \ --period 300 \ --threshold 80 \ --comparison-operator GreaterThanThreshold \ --evaluation-periods 3 \ --alarm-actions arn:aws:sns:us-east-1:123456789012:alerts
CloudWatch Logs Insights Query
Query structured logs for slow requests.
sql
fields @timestamp, @message| filter duration_ms > 1000| sort @timestamp desc| limit 20
Common Tooling
Popular monitoring stacks and services.
- AWS CloudWatch- Native AWS metrics, logs, alarms, and dashboards
- Prometheus- Open-source pull-based metrics collection with its own query language (PromQL)
- Grafana- Open-source dashboarding tool, commonly paired with Prometheus or CloudWatch
- OpenTelemetry- Vendor-neutral standard/SDK for collecting metrics, logs, and traces
- Datadog- Commercial SaaS observability platform covering metrics, logs, APM, and RUM
Advanced PromQL: Rate, Percentiles, and Predictions
Common production PromQL patterns beyond simple gauge lookups.
promql
# Per-second error rate over a 5m window, by servicesum(rate(http_requests_total{status=~"5.."}[5m])) by (service) /sum(rate(http_requests_total[5m])) by (service)# p99 latency from a histogram metrichistogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service))# Predict disk will fill within 4 hours based on the last hour's trendpredict_linear(node_filesystem_avail_bytes{mountpoint="/"}[1h], 4 * 3600) < 0# Alert-friendly: sustained high error rate for 10 minutesmax_over_time( (sum(rate(http_requests_total{status=~"5.."}[5m])) by (service) / sum(rate(http_requests_total[5m])) by (service))[10m:]) > 0.05
OpenTelemetry Collector: Batching, Sampling, Multi-Backend Export
A production-shaped collector config that reduces cardinality/cost before fanning out to multiple backends.
yaml
receivers: otlp: protocols: grpc: http:processors: batch: timeout: 5s send_batch_size: 8192 tail_sampling: policies: - name: errors-always type: status_code status_code: { status_codes: [ERROR] } - name: slow-traces type: latency latency: { threshold_ms: 500 } - name: baseline-sample type: probabilistic probabilistic: { sampling_percentage: 5 } resource: attributes: - key: deployment.environment value: production action: upsertexporters: otlp/tempo: endpoint: tempo.observability.svc:4317 prometheusremotewrite: endpoint: http://mimir.observability.svc/api/v1/pushservice: pipelines: traces: receivers: [otlp] processors: [tail_sampling, resource, batch] exporters: [otlp/tempo] metrics: receivers: [otlp] processors: [resource, batch] exporters: [prometheusremotewrite]
Composite Alarms to Cut Alert Noise
Combine multiple alarms into one page-worthy condition instead of paging on every individual metric breach.
bash
aws cloudwatch put-composite-alarm \ --alarm-name checkout-degraded \ --alarm-rule "(ALARM(high-5xx-rate) OR ALARM(high-p99-latency)) AND ALARM(elevated-traffic)" \ --actions-enabled \ --alarm-actions arn:aws:sns:us-east-1:123456789012:pager-critical \ --ok-actions arn:aws:sns:us-east-1:123456789012:pager-resolved
SLOs, SLIs, and Error Budgets
The SRE vocabulary that turns raw metrics into an alerting and prioritization strategy.
- SLI (Indicator)- A directly measured signal of user experience, e.g. proportion of successful, fast requests
- SLO (Objective)- A target value for an SLI over a window, e.g. 99.9% of requests succeed in under 300ms over 28 days
- Error Budget- The allowed unreliability (1 - SLO); once spent, feature launches pause in favor of reliability work
- Burn Rate- How fast the error budget is being consumed; a burn rate of 10x means the budget empties in 1/10th the window
- Multi-Window Alerting- Alert on both a short window (fast burn, page now) and a long window (slow burn, ticket) to catch both outages and slow leaks
- Cardinality Explosion- Unbounded label values (e.g. user_id as a metric label) that blow up time-series storage and query cost
Structured Logging with Trace Correlation
Emitting JSON logs that carry the active trace/span ID so logs and traces can be pivoted between in one click.
json
{ "timestamp": "2026-07-21T14:32:10.481Z", "level": "error", "service": "checkout-api", "message": "payment provider timeout", "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736", "span_id": "00f067aa0ba902b7", "http.status_code": 504, "http.route": "/v1/checkout", "customer_id_hash": "a13f9c", "retry_count": 2}
Pro Tip
Alert on symptoms (elevated error rate, latency) rather than causes (high CPU) — cause-based alerts create noisy pages for conditions that don't actually affect users.
Was this cheat sheet helpful?
Explore Topics
#CloudMonitoringLogging#CloudMonitoringLoggingCheatSheet#CloudComputing#Intermediate#ThreePillarsOfObservability#CreateACloudWatchAlarm#CloudWatch#Logs#CheatSheet#SkillVeris
