Loki (Log Aggregation) Cheat Sheet
Grafana Loki log aggregation covering LogQL queries, label design, Promtail/Alloy config, and retention tuning.
2 PagesIntermediateFeb 18, 2026
LogQL Basics
Core LogQL syntax for selecting and filtering log streams.
bash
# Stream selector - filter by labels{app="checkout", env="prod"}# Line filter - substring match{app="checkout"} |= "error"# Negative line filter{app="checkout"} != "healthcheck"# Regex line filter{app="checkout"} |~ "timeout|refused"# Parse JSON logs into labels{app="checkout"} | json# Filter on a parsed field{app="checkout"} | json | status_code >= 500# logfmt parsing{app="checkout"} | logfmt | duration > 500ms
Metric Queries (Aggregations)
Turn log streams into time-series metrics for alerting and dashboards.
bash
# Log lines per second matching a filtersum(rate({app="checkout"} |= "error" [5m]))# Rate grouped by labelsum by (pod) (rate({app="checkout"}[5m]))# Count over timecount_over_time({app="checkout"} |= "panic" [1h])# Extract a numeric field and compute quantile latencyquantile_over_time(0.99, {app="checkout"} | json | unwrap duration [5m]) by (route)# Bytes ingested per streamsum(bytes_rate({app="checkout"}[5m])) by (pod)
Promtail / Grafana Alloy Scrape Config
Minimal Promtail config to ship container logs to Loki with pipeline stages.
yaml
server: http_listen_port: 9080clients: - url: http://loki:3100/loki/api/v1/pushscrape_configs: - job_name: kubernetes-pods kubernetes_sd_configs: - role: pod pipeline_stages: - docker: {} - json: expressions: level: level msg: message - labels: level: relabel_configs: - source_labels: [__meta_kubernetes_pod_label_app] target_label: app - source_labels: [__meta_kubernetes_namespace] target_label: namespace
Key Loki Server Config Options
Common limits_config and storage settings you'll tune in production.
- retention_period- how long chunks are kept before compactor deletes them, set in limits_config
- ingestion_rate_mb- per-tenant ingestion rate limit; raise for high-volume tenants
- max_query_series- caps how many streams a single query can touch, protects against runaway queries
- split_queries_by_interval- breaks large time-range queries into parallel chunks for the query-frontend
- chunk_encoding- compression codec for chunks (snappy, gzip, zstd); zstd gives best ratio
- boltdb-shipper / tsdb- index store types; tsdb is the modern recommended index for Loki 2.9+
Ruler: Alerting & Recording Rules
Loki's ruler evaluates LogQL as Prometheus-style alert and recording rules without a separate eval engine.
yaml
# ruler/rules/checkout-alerts.yamlgroups: - name: checkout-errors rules: - alert: HighErrorRate expr: | sum(rate({app="checkout"} |= "error" [5m])) / sum(rate({app="checkout"}[5m])) > 0.05 for: 10m labels: severity: page annotations: summary: "checkout error ratio above 5%" - record: checkout:requests:rate5m expr: sum(rate({app="checkout"}[5m])) by (route)
Multi-Tenancy with X-Scope-OrgID
Loki isolates data per tenant using a required org header once auth_enabled is turned on.
bash
# Query tenant 'team-checkout' directly against Loki's HTTP APIcurl -s -G \ -H "X-Scope-OrgID: team-checkout" \ --data-urlencode 'query={app="checkout"} |= "panic"' \ http://loki:3100/loki/api/v1/query_range# Promtail must also stamp the same header when auth_enabled: true# (set in the client block, not the scrape config)clients: - url: http://loki:3100/loki/api/v1/push tenant_id: team-checkout
Object Storage Backend + Schema Config
Production Loki ships chunks and the TSDB index to object storage using a versioned schema_config.
yaml
schema_config: configs: - from: 2024-01-01 store: tsdb object_store: s3 schema: v13 index: prefix: index_ period: 24hstorage_config: aws: s3: s3://us-east-1/my-loki-bucket s3forcepathstyle: false tsdb_shipper: active_index_directory: /loki/tsdb-index cache_location: /loki/tsdb-cachecompactor: working_directory: /loki/compactor delete_request_store: s3 retention_enabled: true
Advanced LogQL: pattern, label_format, unwrap
Reshape and re-label log streams mid-query instead of only filtering them.
bash
# Extract fields with a lightweight pattern (faster than regex for fixed formats){app="nginx"} | pattern `<ip> - - [<ts>] "<method> <path> <_>" <status> <size>`# Re-derive a label from parsed fields for grouping{app="checkout"} | json | label_format route_group="{{.route}}"# Rewrite the display line itself{app="checkout"} | json | line_format "{{.level}} {{.msg}}"# unwrap + math on a numeric field, bucketed rate of changesum by (pod) ( rate( {app="checkout"} | json | unwrap bytes_sent | __error__="" [5m] ))
Query-Frontend & Caching Knobs
Fields that determine how large time-range queries get split, parallelized, and cached.
- split_queries_by_interval- chunks a wide query into per-interval sub-queries executed in parallel by queriers
- parallelise_shardable_queries- lets the frontend shard aggregations (sum/count/rate) across queriers by stream
- results_cache / cache_results- caches finished query results so repeated dashboard panel refreshes hit cache, not storage
- max_retries- how many times the frontend retries a failed sub-query before surfacing an error
- query_timeout- upper bound before a query is aborted, tune per tenant via overrides
- max_query_parallelism- caps concurrent sub-queries a single request can fan out to, protects querier CPU
Pro Tip
Never put high-cardinality values (user IDs, request IDs, trace IDs) in Loki labels — put them in the log line and query with `| json | field="x"` instead, since every unique label combination creates a separate stream and blows up your index.
Was this cheat sheet helpful?
Explore Topics
#LokiLogAggregation#LokiLogAggregationCheatSheet#DevOps#Intermediate#LogQLBasics#MetricQueriesAggregations#Promtail#Grafana#Databases#CheatSheet#SkillVeris
