Five things you can do with Sentry's Agent Tracing

Agent failures are often silent: a guessed parameter, a retry loop, a bill that creeps up. Sentry's Agent Tracing puts model calls, tool calls and handoffs in one trace next to the rest of the request, and this post collects five recipes built on it. Real production conversations become an eval dataset, exported with the Sentry CLI and scored with cheap deterministic checks (did it cite real records, did it stay under a token budget) before any LLM judge. A daily Claude routine reads overnight conversations through the Sentry MCP server and files Linear tickets for new tool failures and loops. The other three cover per-call cost tracking with the Vercel AI SDK, per-user spend dashboards with hourly alerts, and adding Sentry to an existing OpenTelemetry pipeline. The cautions are the useful part: parent agent spans include their children, so summing raw spans double-counts cost; Edge routes send no AI spans; and on AI SDK 7, skipping registerTelemetry silently drops them. Note the tradeoff, too. Several recipes depend on Sentry 11 recording prompts and outputs by default, which this site turns off on purpose so visitor conversations with our concierge stay out of error tracking. It is a vendor post, but the recipes are concrete and the example code is public.