Journal

I run my company's software on agent fleets every day. They almost never stall on the model. They stall on four operating problems that compound: reliability, observability, context, and cost. Here's the map, from someone who hits these in production.

Most failed agent deployments in DeployBench weren't crashes. 97 of 154 failures were the agent deciding it was finished after validating a weaker target than the task required. The expensive failure mode is completion judgment, and the fix is a verifier the agent can't redefine.

When an agent run is going to fail, it usually shows signs early, then keeps spending. A 2026 study found that on warned failed runs, 58.1% of tokens were spent after the first warning. That's compute you paid for on a run you could have already known was doomed.

Agent memory looks solved until you put it in a real group chat. A 2026 benchmark found the strongest memory system scored 46.0% average accuracy, and a plain BM25 keyword search matched or beat most of them. Memory is a retrieval-quality problem, not a vector DB you bolt on.

Observing AI agents isn't a dashboard of token counts. It's watching the execution trace for the recurring ways agents fail, attributing a failure to the step that caused it, and catching a doomed run before it finishes spending. Here's the discipline, organized by how agents actually fail.

The most accurate multi-agent architecture is usually not the one to put in production. In a 2026 benchmark on financial document extraction, the most accurate setup cost 2.3x the baseline, while a hybrid recovered 89% of its gains at 1.15x. Architecture is a cost-accuracy choice, not a leaderboard.

A multi-agent system fails and you have to find the one step that caused it. If your traces only logged what each agent said, not what it saw, you mostly can't. A 2026 benchmark found full traces improve failure attribution by up to 76% over output-only logs.

The agent bill surprises teams, and the model price is rarely the reason. Most of the cost is context you pay to reprocess on every call: the same docs reread each session, the whole history carried forward. AI isn't mainly a cost-cutting play, but the waste inside the bill is real and mostly avoidable.

The success number from a demo is a single lucky run. A 2026 benchmark stressed agents the way production does, with task perturbations and injected tool failures, and watched success fall from 96.9% to 88.1% on perturbation alone. Reliability is a surface you measure, not a number you quote.

Reliability isn't a model property you buy, it's an operating discipline you build. Demo success rates predict almost nothing: agents self-correct poorly, report false success, and degrade under load. The fix is gating on verified signals, not the agent's own word. Here's the discipline, with the evidence.

Agents report tasks as done that aren't. A June 2026 study found false success in up to 75.8% of self-assessing coding-agent runs, and the LLM judge you'd use to catch it scored near a coin flip. The fix is gating on verified state, not self-report.

Context engineering is deciding what an agent reads at each step instead of feeding it everything. Across 2026 studies the pattern holds: more context usually makes agents slower, costlier, and less accurate. A bigger window doesn't fix it. Here's the discipline, and where each piece fits.

The reflex is to give an agent more context when it struggles. A June 2026 benchmark shows the opposite: keeping the full conversation history scored 71%, while pruning and summarizing hit 91.6%, on a third of the tokens. Selecting context beats hoarding it.

Telling an agent to double-check its own work mostly doesn't work. A June 2026 study shows why: the same wrong claim gets corrected the moment it's labeled as someone else's. Self-review is a prompt-format blind spot, not a missing skill.

We run a fleet of autonomous agents that are the first real users of our own dev tools. So we interviewed them. Agents confabulate, so a complaint only counted when the telemetry backed it up. What survived is our roadmap, and a lesson in designing for agents as users.

For a Swiss avocat, professional secrecy is a criminal duty that outlives your practice. Whether an AI tool breaches it is not decided by the brand or the licence tier. It is decided by three terms in the vendor's Data Processing Addendum, and by one Swiss trap no data-protection clearance can fix.

The same agent task can cost 30x more on one run than another, spending more tokens does not buy more accuracy, and the model itself cannot tell you the bill before it runs. A 2026 study across 8 frontier models on SWE-bench Verified put numbers on all three. The cost is not high so much as it is unpredictable.