100% Free Forever
AI-Powered Learning
Industry Expert Content
Certificates & Badges
Learn At Your Own Pace
Agent Evaluation & Observability
30 minadvanced

Instrumenting an Agent with OpenTelemetry

Every observability vendor wants to be the place your traces live, which means every vendor ships its own SDK, its own span format, and its own idea of what an agent's LLM call attribute should be called. Instrument an agent directly against one vendor's SDK and you have coupled your entire agent codebase to that vendor's roadmap, pricing, and outages. Switching later means re-instrumenting every span in every tool call, not swapping a config value.

OpenTelemetry (OTel) exists to break that coupling. It is a CNCF-hosted, vendor-neutral specification plus a set of language SDKs for producing traces, metrics, and logs in a standard shape. You instrument your code once, against the OTel API, and then choose where the data goes — Honeycomb, Datadog, Arize Phoenix, an open-source Jaeger instance, or, as in every example in this lesson, nowhere at all except your own terminal — purely by swapping an exporter. The instrumentation in your agent loop does not change.

This lesson is about the mechanics of producing that data: the tracer/provider/exporter/processor model that every OTel SDK shares, the GenAI semantic conventions that make an LLM call span mean the same thing across every team that emits one, and context propagation, which is how a span created three function calls deep inside a tool handler still ends up correctly nested under the agent turn that triggered it. The shape of a trace as a whole — how spans compose into a tree that represents one agent run end to end — is the next lesson. Here, the concern is narrower: how do you get a span onto the wire at all, correctly, without blocking your agent or leaking data you shouldn't record?

Analogy🏏Cricket
🏏 Think of it like cricket: every major tournament — the IPL, the Ashes, a Ranji Trophy final — is filmed once, at the ground, by a single production truck that captures the 'world feed': raw camera angles, Hawk-Eye ball-tracking data, stump-mic audio, all in a standard broadcast format. Star Sports in India, Sky Sports in the UK, and Fox Cricket in Australia don't each install their own cameras behind the bowler's arm at the MCG — they all plug into that same world feed and lay their own commentary, graphics, and ad breaks on top of it. If Sky Sports one day decides ball-tracking data is too expensive and switches to a cheaper analytics partner, nothing at the ground changes; they just point their broadcast van at a different feed. Now imagine the alternative: if the stadium had wired its cameras directly into Sky Sports' proprietary graphics box, switching broadcasters would mean re-running cable through the entire ground. OpenTelemetry is the world feed for your agent's telemetry — you wire your agent loop to it once, in a standard format, and any 'broadcaster' — Honeycomb, Datadog, a console printout — can plug in downstream without you touching the instrumentation again. The insight: standardizing the capture format is what makes the distribution layer swappable.
Lesson 19 of 35
0% complete