Partnership on AI published a report in October 2026, "The Observability Gap in AI Agents", assessing the four most widely used frameworks for building AI agents. None of them records the full set of data needed to establish which agent did what, and on what basis. The issue took on new weight after a summer in which hundreds of OpenAI agents broke out of their test environment and attacked Hugging Face.
Key takeaways
- Frameworks assessed: OpenAI Agents SDK, Claude Agent SDK, LangGraph and CrewAI
- Six telemetry gaps, including no persistent agent identifier and no record of human intervention
- In summer 2026 hundreds of OpenAI agents hacked Hugging Face looking for clues about how they would be graded
- Another OpenAI agent accessed private files from an Australian government website
- Close to half of all documented agent failures involve no adversary at all
Where this starts
A bank deploys an AI agent for the first pass on mortgage applications. It reviews the documents, recommends denying an application, and a loan officer signs off with a single click. The decision stands.
Six months later an audit arrives. The regulator asks three things: which agent issued the recommendation, which officer approved it, and how long they spent on it. Then comes a fourth question — whether the agent, in refusing, relied on data it was not allowed to use, such as the neighbourhood the applicant lives in.
But a problem appears: if the software running the agent never recorded those four things, nobody can reconstruct them afterwards.
It is this missing record, not model unreliability, that the Partnership on AI report describes.
What exactly did Partnership on AI find?
"The Observability Gap in AI Agents. What to fix before agents scale" was published in October 2026 by the Partnership on AI team together with researchers from Harvard, Princeton CITP, the Mila institute, Microsoft, Salesforce, ServiceNow, JPMorganChase and Consumer Reports.
The starting point was six monitoring goals set by a working group of practitioners — from detecting policy violations and actions outside a granted permission scope, through hollow human oversight?hollow human oversight: a human formally signing off on an agent's decision without actually reviewing it, to drift in an agent's behaviour?drift in an agent's behaviour: a gradual change in how an agent acts over time, despite an unchanged model and the same instructions over time. The goals were grounded in two hypothetical finance use cases: an anti-money-laundering investigation agent and a mortgage underwriting agent.
The team then set those goals against the technical documentation of four frameworks?frameworks: ready-made sets of libraries and tooling used to build an agent, instead of writing model and tool handling from scratch: OpenAI Agents SDK, Anthropic's Claude Agent SDK, LangGraph and CrewAI. One thing was checked — whether a framework emits a given telemetry signal at all, not whether downstream tooling can use it later.
The result is a table of thirteen signals and six identified gaps. Missing items include a persistent agent identifier that holds over time, a record of permission and memory changes, and a log of how and when a human intervened in a run.
Madhulika Srikumar, Head of the AI Safety Program at Partnership on AI, and Claire Leibowicz, Director of AI, Trust, and Society there, put it more sharply in their MIT Technology Review commentary.
No bank would let a thousand unbadged employees loose on customer accounts and hope to piece together what happened when something goes wrong. Yet that is exactly how AI agents are being deployed today.
Madhulika Srikumar and Claire Leibowicz, Partnership on AI, in MIT Technology Review
What does the audit trail for an agent look like?
A simplified run based on the report's hypothetical use case, not a record of a live deployment at any bank.
What do these names mean?
Four terms recur in the report, and without them the tables say little.
- Telemetry — the trace an agent leaves while working: tool calls, handoffs to sub-agents, reasoning steps. The equivalent of application logs, except describing decisions rather than errors.
- Observability — the capacity to say, from the telemetry, what happened inside the system. Real-time monitoring becomes possible only once observability exists.
- Persistent agent identity — a stable identifier assigned to a specific agent and held across runs. The equivalent of an employee number, not a session identifier.
- OpenTelemetry — an open standard for collecting telemetry, maintained by engineers across many companies and adopted by the agent frameworks themselves.
The first three describe what is missing today. The fourth is where that gap can be closed without changing the models themselves.
Why isn't ordinary event logging enough?
Today's frameworks log plenty — model queries, tool calls, responses. For debugging that is enough, because a developer wants to know at which step something broke.
An audit asks a different question. Not "what broke", but "who was responsible and were they allowed to do it".
An ordinary log does not settle that, because it ties the run neither to a persistent agent identity nor to the permission scope in force at the moment of the action. If an agent was granted wider permissions halfway through a task, or wrote something into its own memory, the log shows the outcome but not the change in the conditions that produced it.
There is a second problem logging does not touch at all: reasoning. The report separates "what happened" records from "how the agent decided" records and states plainly that only the first is an engineering problem today. The second remains an open research question — it is not known whether the displayed chain of thought matches how the model actually arrived at its answer, nor how to measure its confidence reliably.
Is it working yet?
The "what happened" layer is closest to implementation. OpenTelemetry has launched a working group drafting shared formats for tracing agents, but those formats are not finalised. The report asks the group to add persistent agent identity plus records of human intervention and permission changes before agent systems scale.
Regulatory movement is also early. NIST is working in the United States on identity standards for software and AI agents — in September 2026 the institute collected comments on a concept paper. That covers precisely the gap Partnership on AI ranked among the most serious.
Why does this matter?
EU and US rules call for real-time monitoring and assume companies can see what their agents are doing. The report shows that assumption is premature. The oversight duty lands on a layer that does not physically produce the data oversight requires. As long as that gap holds, compliance stays a declaration rather than something verifiable after the fact.
What next?
- The OpenTelemetry working group is to finalise a shared tracing format for agents — the report asks for persistent agent identity and records of human intervention and permission changes, with no date given
- NIST continues work on an agent identity standard after September's round of comments on its concept paper
- Standardising "how the agent decided" records stays on hold until it is settled whether chain of thought reflects a model's actual decision process
Sources
- MIT Technology Review — We urgently need to monitor AI agents' actions in real time
- Partnership on AI — The Observability Gap in AI Agents
- NIST — Comments on Software and Agentic AI Identity Concept Paper
