New Relic Cheat Sheet
Reference for New Relic APM agent setup, NRQL query syntax, and alert condition configuration for application performance monitoring.
NRQL Basics
Common NRQL query patterns against APM event data.
-- Average response time for a transaction, last 30 minSELECT average(duration) FROM TransactionWHERE appName = 'checkout-service' SINCE 30 minutes ago-- Error rate grouped by time bucketSELECT percentage(count(*), WHERE error IS true) FROM TransactionTIMESERIES 5 minutes-- Top 5 slowest transactionsSELECT average(duration) FROM TransactionFACET name ORDER BY average(duration) DESC LIMIT 5
Agent Configuration
newrelic.yml key settings for a Node.js/Python agent.
common: &default_settings license_key: <YOUR_LICENSE_KEY> app_name: checkout-service distributed_tracing: enabled: true logging: level: infoproduction: <<: *default_settings
Key Concepts
Core New Relic terminology.
- APM agent- Language-specific library instrumenting an app to report transactions/traces
- NRQL- SQL-like query language for querying New Relic's event/metric data
- entity- A monitored object (app, host, service) with its own health and dashboards
- alert condition- A threshold on an NRQL query or metric that triggers a violation/notification
- distributed tracing- Correlates spans across services for a single request into one trace
NRQL Alert Condition
A static threshold alert condition definition.
{ "nrql": { "query": "SELECT average(duration) FROM Transaction WHERE appName = 'checkout-service'" }, "terms": [ { "threshold": "2", "thresholdOccurrences": "ALL", "thresholdDuration": 300, "operator": "ABOVE", "priority": "CRITICAL" } ]}
NRQL Subqueries & Anomalies
Nested NRQL subqueries and anomaly-style comparisons for deeper transaction analysis.
-- Compare current error rate against baseline from 1 week agoSELECT percentage(count(*), WHERE error IS true) AS 'errorRate'FROM TransactionWHERE appName = 'checkout-service'SINCE 30 minutes ago COMPARE WITH 1 week ago-- Subquery: hosts whose error count exceeds the fleet averageSELECT hostname, errorCount FROM ( SELECT count(*) AS errorCount FROM Transaction WHERE error IS true FACET hostname SINCE 1 hour ago) WHERE errorCount > ( SELECT average(errorCount) FROM ( SELECT count(*) AS errorCount FROM Transaction WHERE error IS true FACET hostname SINCE 1 hour ago ))-- Histogram of transaction durationsSELECT histogram(duration, 10, 20) FROM TransactionWHERE appName = 'checkout-service' SINCE 1 hour ago
Custom Instrumentation (Node.js API)
Adding custom spans, attributes, and events with the New Relic Node.js agent API beyond auto-instrumentation.
const newrelic = require('newrelic');// Record a custom event with searchable attributesnewrelic.recordCustomEvent('CheckoutCompleted', { cartValue: 129.99, itemCount: 3, paymentMethod: 'card',});// Wrap an async block in a custom segment for tracingasync function processOrder(order) { return newrelic.startSegment('processOrder', true, async () => { newrelic.addCustomAttribute('orderId', order.id); return await chargeCard(order); });}// Explicitly ignore noisy background transactionsnewrelic.setIgnoreTransaction(true);// Propagate a distributed trace header manually across a non-HTTP boundaryconst headers = {};newrelic.getTraceMetadata(); // { traceId, spanId, ... }newrelic.insertDistributedTraceHeaders(headers);
Alert Policy as Code (Terraform)
Provisioning a baseline (anomaly) alert condition and notification workflow via the newrelic Terraform provider.
resource "newrelic_alert_policy" "checkout" { name = "checkout-service-policy" incident_preference = "PER_CONDITION_AND_TARGET"}resource "newrelic_nrql_alert_condition" "latency_anomaly" { policy_id = newrelic_alert_policy.checkout.id type = "baseline" name = "Latency Anomaly" baseline_direction = "upper_only" violation_time_limit_seconds = 3600 nrql { query = "SELECT average(duration) FROM Transaction WHERE appName = 'checkout-service'" } critical { operator = "above" threshold = 3 threshold_duration = 300 threshold_occurrences = "ALL" }}resource "newrelic_notification_channel" "slack" { name = "checkout-alerts-slack" type = "WEBHOOK" product = "IINT" destination_id = newrelic_notification_destination.slack.id channel_type = "WEBHOOK"}
Advanced Concepts
Terminology beyond basic APM setup, relevant to fleet-scale observability.
- baseline alert condition- Dynamically adapts thresholds to a metric's historical seasonal pattern instead of a fixed static value
- NRDB- New Relic's telemetry data store; all NRQL queries execute against it, dimensional metrics + events + logs + traces
- span attributes vs custom attributes- Span attributes are scoped to one trace segment; custom attributes attach to the whole transaction event
- entity synthesis- Rules that turn incoming telemetry (tags/attributes) into first-class monitored entities in the UI
- drop filters (NRQL drop rules)- Ingest-time rules that discard or obfuscate matching data before it's billed and stored
- workload- A logical grouping of entities (e.g. all services for one product) with an aggregated status rollup
- APM 8T (backend) vs Browser vs Mobile agents- Separate agent families reporting into the same entity model but with different auto-instrumentation
Logs in Context & Log Patterns
Correlating logs with traces and mining recurring log patterns via NRQL.
-- Find logs correlated to a specific traceSELECT timestamp, message, level FROM LogWHERE trace.id = 'a1b2c3d4e5f6'SINCE 1 hour ago-- Surface the most frequent error log clustersSELECT count(*) FROM LogWHERE level = 'ERROR'FACET aparse(message, '* failed with *') AS reasonSINCE 1 day ago LIMIT 10-- Log forwarding config snippet (newrelic-infra logging.yml)-- logs:-- - name: checkout-service-- file: /var/log/checkout/app.log-- attributes:-- logtype: json
Use FACET with TIMESERIES together in NRQL to spot which specific transaction or host is driving an aggregate spike, rather than alerting only on overall averages that can hide localized problems.