AI Agent Observability: What Your Stack Runs Without You

TL:DR: AI agent observability is mostly a memory problem. Your agent sets up a cron job, a systemd timer, and three MCP servers over six weeks, then forgets all of it, because none of it is in the context window. Vitals is a local-first tool that reads what is actually configured on your machine and hands the agent a summary before it touches anything. It is GPL v3, early, and still moving.

Give an agent enough autonomy and it will start scheduling things. A cron job to scrape a feed. A systemd timer for backups. An MCP server you wired up in March and have not thought about since.

Six weeks later you ask it to "clean up the old scraper" and it has no idea what exists. Not because it is dumb, but because none of that is in the context window. Every session starts blind. The agent that set the cron job up is not the agent you are talking to now.

That is the gap agent-vitals is trying to close. It is early, I am still working on it, and this post is as much a description of the problem as a pitch for the tool.

What "AI agent observability" means when there is no cluster

Observability normally means dashboards, traces, and a service to send them to. That framing does not fit here.

A solo developer running Claude Code, Cursor, and a couple of agents does not have a distributed system. They have a laptop with an accumulating pile of state: crontabs, systemd units, MCP server configs across four different hosts, skill files, and session logs. None of it is centralised, and all of it acts on their behalf.

The questions are not "what is my p99 latency". They are:

1. What is scheduled to run without me?
2. What is quietly broken or pointing at a deleted script?
3. What is my agent doing repeatedly that is not working?
4. What is all of this costing in tokens?

Nobody needs Grafana for that. You need something that reads local state and tells the truth about it.

The design constraint: the agent is the reader

This is the part that shaped everything else. Vitals is not primarily a CLI for you to read. It is a tool your agent calls from inside its own loop.

It exposes 13 tools over the Model Context Protocol, so the agent can ask questions directly: vitals_summary, shadow_list, shadow_stale, burnout_summary, burnout_stuck_sessions, plus eight tools for session traces.

The one that matters most is vitals_summary, because it is designed to be cheap enough to call at the start of every session. Here is what it actually returns:

  ▄▀ vitals - local agent stack health

  shadow: 13 autonomous thing(s) configured (mcp: 8, systemd: 5)
  subagent burnout (7d): 5 runs, 100% completion ✓
  claude code (7d): 170 sessions, 45,626 events, ⚠ 36 stuck
    - biggest stuck-looking session: 7,118 events

Four lines. That is the whole point. A summary that costs a paragraph of context is one an agent can afford to call before it does anything risky. A summary that costs two thousand tokens is one it will skip.

The install step also drops a priming skill file, so the agent knows when calling vitals is appropriate rather than waiting to be told.

What it looks for

The CLI has 20 commands. They cluster into three questions.

What is running? av shadow enumerates everything scheduled or configured to act on your behalf, across cron, systemd, MCP servers, and skills. av doctor is a one-shot health check with recommendations. av drift finds inconsistencies across your agent hosts, where Cursor has an MCP server that Claude Code does not, and so on.

What is wasting? av loops detects doom-loop patterns, meaning the same Bash command run twenty or more times, or the same edit applied ten or more times with no progress. It compares edits by content and excludes legitimate polling, because a watch loop is not a doom loop. av ssh catches repeated SSH commands to the same host. av unused finds MCP tools that are registered but never called, which still cost you context on every single turn. av overlap finds duplicate tool names across servers.

What does it cost? av cost parses session JSONLs and groups spend by project. av burnout reports completion rates and stuck-session heuristics. av trace is session forensics for Claude Code and pi, with no payloads and no secrets stored.

The stale detection is the one that earns its place fastest. A systemd unit pointing at a script you deleted two months ago does not announce itself. It just fails quietly forever.

The hooks are the opinionated part

Reading state is easy. Getting an agent to read it before acting is the hard part, and it is the piece I am least sure about.

Vitals can install PATH wrappers around crontab and systemctl that gate mutations until vitals data is fresh. If the agent tries to edit your crontab against a stale picture of the machine, the wrapper stops it and tells it to refresh first.

This is a genuinely intrusive thing for a tool to do, which is why it is off by default. av install defaults to three components: the MCP server, the priming skill, and a snapshot of current state taken before anything changes. Hooks, doom-loop detection, unused-tool detection, and cost tracking are all opt-in from the same menu.

I would rather ship the safe default and let people turn on the sharp parts deliberately. That is the same reasoning behind taking a state snapshot before install: if the tool makes things worse, you can get back.

Local-first, and what that actually buys

There is no daemon, no cloud, and no account. Everything is read from files that are already on your disk: crontabs, systemd units, MCP config files, and session JSONLs.

This is not a privacy flourish. It is the only design that makes sense for the data involved. Session logs contain your prompts, your file paths, and often your business logic. Shipping that to a hosted backend so you can see a chart about it is a bad trade. The same argument we make about owning your business data applies with more force to the logs of your development work.

It also keeps the tool cheap to reason about. It is roughly 6000 lines of Python with 295 tests, installed with uv:

uv tool install agent-vitals
av install    # detects hosts, wires MCP + priming skill

It detects pi, Claude Code, Cursor, OpenCode, and Codex CLI, and wires up whichever ones you actually have. Restart the host afterwards so it picks up the new MCP server.

Where it is honestly incomplete

This is version 0.7.0 and I am still working on it. Being clear about the state of things:

• The stuck-session heuristic is a heuristic. It flags sessions with unusually high event counts. That correlates with sessions going wrong, but a genuinely long session gets flagged too. It is a hint, not a verdict.
• Host coverage is uneven. Trace forensics work properly for Claude Code and pi. The other hosts get shadow and drift detection but not the deeper session analysis, because their log formats differ.
• Cost estimates depend on a pricing table that has to be kept current as models change. It will drift.
• The hooks approach is unproven. Gating crontab behind a freshness check is a strong opinion. It might turn out to be more annoying than useful at scale.
• Scanners are Linux-first. systemd unit scanning does nothing useful on macOS, where launchd is the equivalent and is not yet covered.

None of that is a reason not to use it, but it is the difference between a tool I use daily and one I would call finished. TuxSpeak is finished in a way vitals is not.

FAQ

Does this send my session logs anywhere?

No. Every command reads local files and prints locally. There is no telemetry, no backend, and no account. The trace tooling explicitly stores no payloads and no secrets, only structural information about what happened.

Why MCP instead of just a CLI?

Because the intended reader is the agent, not you. A CLI requires a human to remember to run it. An MCP tool plus a priming skill means the agent can check its own infrastructure at the start of a session, which is exactly when the information is useful and exactly when a human would not have thought to look.

What is a doom loop, specifically?

Two patterns. Exact repetition, where the same Bash command string runs twenty or more times, or the same edit with identical old and new content is applied ten or more times. And softer variants where the operation is nearly identical. Polling is excluded, since a loop that is deliberately watching for a change is doing its job.

Do I need to install the hooks?

No, and they are off by default. The default install is the MCP server, the priming skill, and a state snapshot. Hooks, loop detection, unused-tool detection, and cost tracking are opt-in, because gating crontab and systemctl is an intrusive thing to do to someone's machine.

Does it work with agents other than Claude Code?

It detects and wires pi, Claude Code, Cursor, OpenCode, and Codex CLI. Shadow scanning and drift detection work regardless of host, since they read system state rather than agent logs. The deeper session analysis currently only understands Claude Code and pi log formats.

Is it production ready?

It is at 0.7.0 with 295 tests, and I run it on my own machine daily. But the heuristics are still being tuned and the API is not frozen. Use it, and expect the flags and output to move.

Bottom line

The interesting problem in AI agent observability right now is not tracing. It is that agents accumulate infrastructure across sessions and have no memory of doing it. The state is real, it runs without you, and it is invisible to the thing that created it.

Vitals is one answer: read the local state, summarise it in four lines, expose it over MCP, and let the agent check before it acts. Whether the hooks idea survives contact with real usage is still an open question, and I will write about it either way.

It is on GitHub under GPL v3. The same preference for tools you own and can read runs through everything else we build, and through our argument for a simpler stack generally. The features page covers the rest of the toolkit, compare puts it next to the alternatives, and the FAQ handles the obvious objections.