When a company hesitates to put an AI agent into production, the worry is almost always about capability: "will it be good enough?", "will it hallucinate?"
Wrong question. An agent that fails loudly gets caught. The real problem is the agent that fails silently - the one that keeps returning answers that look right. Datadog puts it bluntly in its *State of AI Engineering 2026* report: "the next wave of agent failures won't be about what agents can't do, but about what teams can't observe".
And Datadog happens to be well placed to say it: that is exactly the problem they solved fifteen years ago, on different ground.
The problem isn't new, it just changed targets
Datadog was founded on 4 June 2010 around a very simple observation: between the developers writing the code and the teams running it, there was a "wall of confusion". Everyone saw a piece of the system, nobody saw the system. When things broke, people passed the blame around for lack of a shared view.
Their bet: do not fragment. Where the market was selling one tool per symptom, Datadog brought the three pillars of observability - metrics, traces and logs - into a single product, as co-founder Olivier Pomel recounts on Matt Turck's MAD podcast. An expensive bet, slower to build, harder to sell than a quick point fix. The market settled it: at its IPO in September 2019, CNBC reported, the stock jumped from 27 to 40.50 dollars on day one, valuing the company at 10.9 billion.
Today, the same wall of confusion is going back up - between the company and its agents.
What it looks like when the box is black
An agent without observability is a plane without a black box. You know it took off, you know it crashed, and you have no way of knowing why. No record of the reasoning, no record of the decision, no record of the tool called at 3 a.m.
The numbers describe a fleet of planes with no black boxes. Datadog estimates that roughly 1 in 20 AI requests already fails in production, nearly 60 % of them for capacity reasons - while the system keeps running and keeps returning outputs that look fine. An IBM study of 2,000 technology executives counts 54 agent incidents per organisation on average over the year, 17 % of them high severity - more than four hours to contain. And according to Shakudo, mean time to resolution goes from under an hour to 4.2 hours when observability is missing: more than four times longer, for the same outage.
The concrete cases speak for themselves. Zenity describes how a coding agent wiped a production database in 9 seconds: blocked by an invalid credential, it went looking for an API token in an unrelated file, then triggered a volume deletion. Elsewhere, OpenLeash reports, an agent destroyed a database holding more than 1,200 executive records in the middle of a code freeze - a freeze it had explicitly acknowledged. In both cases, it wasn't the agent's capability that failed. It was the record of the decision.
Agent observability, to be precise
The technical term is observability - and applied to agents, it covers three concrete things.
- Traces: the full tree of an execution, step by step (every model call, every tool invoked, every sub-agent triggered, with its inputs and outputs) - what are known as spans.
- Metrics: latency, cost, error rate, token consumption, per agent and per step.
- Logs: the raw textual record, including the prompt actually sent - not the one you think you wrote.
Without those three layers, as Galileo puts it, "an agent system is a black box: you know what went in and what came out, but not why a given decision was made, which step introduced the error, or whether the system is slowly degrading". That is exactly what's missing when you need to answer the only question that matters on incident day: prompt, model, tool or orchestration?
Why this is urgent now
Because complexity just crossed a threshold. Again from the Datadog report, adoption of agent frameworks nearly doubled in a year, going from around 9 % to close to 18 % of organisations, and 69 % of companies are already running three or more models in tangled workflows. Datadog responded by extending its founding logic to agents: end-to-end traces, testing, centralised governance of both in-house and third-party agents.
The bill for the alternative is already priced. Gartner expects more than 40 % of agentic AI projects to be abandoned by the end of 2027, notably because of runaway costs and insufficient risk control. And an MIT study put at 95 % the share of generative AI pilots with no measurable impact on the bottom line. Both numbers say the same thing: you can't make decisions about what you don't measure - not the value created, not the real cost per task.
What we do
At Elephant in the Room, we instrument agents from day one, not after the first incident: every run traced step by step, cost and latency attributed to the agent consuming them, guardrails on destructive actions, and a human in control where it matters. Those traces then feed back into the company brain - every failure becomes reusable context instead of a lost lesson.
That's what separates an AI-native company from a company that "added some AI" somewhere.
An agent you can't observe isn't a colleague: it's a bet.
