AI agents finished a third of the real work. The demo skips that part.
ORAgentBench gave 14 frontier agent setups 107 expert-reviewed operations-research tasks with real briefs, data, and constraints. The best one finished 35.5%, and 20.6% of the hard tasks. The failures weren't reasoning. They were the operational discipline a demo never has to show.
tsukumo
Short version: the agent demo always works. It picks a clean task, runs to a green checkmark, and the room nods. The question that demo never answers is what happens on real work, with real constraints, graded by someone who knows the domain. A 2026 benchmark called ORAgentBench went and checked, on 107 expert-reviewed operations-research tasks, across 14 frontier agent setups. The best one finished about a third. On the hard tasks, a fifth. Read that before you put an agent on your critical path.
ORAgentBench (Li, Cai, Li et al., 2026) is a benchmark for a plain question its title asks directly: can LLM agents solve challenging operations-research tasks end to end? Operations research is a good stress test, because the work is unforgiving. A solution is either feasible against the constraints or it isn't, and "looks right" earns nothing.
The setup is the part that makes the result credible:
107 human-reviewed tasks, each in an isolated environment with a natural-language brief, multi-file data, and configuration artifacts. Real inputs, not toy prompts.
14 frontier agent-model combinations tested, so this isn't one weak agent dragging the average down.
Every submission validated for schema compliance, constraint feasibility, and solution quality, rather than only whether it ran.
The headline: the best configuration scored 35.5% overall, and 20.6% on the hard tasks. Plenty of agents produced submissions that were feasible but landed below the quality bar. They finished. The work just wasn't good enough.
This is where it gets useful for anyone deploying agents, because the obvious read, "the models aren't smart enough yet," is the wrong one. The authors traced the errors, and they cluster somewhere specific: missed operational rules, brittle problem formulations, and weak construction and improvement of feasible solutions.
Look at that list. None of it is a reasoning puzzle the model couldn't crack. It's the discipline of real work. The agent skips a constraint buried in the brief. It commits to a fragile framing of the problem and never reconsiders. It finds a feasible answer and stops, instead of pushing it toward a good one. A strong senior wouldn't do those things, not because they reason better in the abstract, but because they know the operational rules and they don't accept their own first feasible draft.
Spotting exactly this, an agent that produces plausible output that quietly violates a domain rule, is part of what an agent-ops assessment looks for before that output reaches anything that matters.
The finding that should stop the "just prompt it better" plan#
There's one result in the paper worth pinning to the wall. The authors tested adding OR-specific procedural skills, the obvious fix, more domain scaffolding. It raised hard-task feasibility. It did not reliably improve solution quality or pass rate. Their own framing: progress requires moving beyond plausible optimization code toward dependable, high-quality operational decision-making.
So the easy lever, bolt on more domain prompting, moved the cheap metric (did it produce something valid-shaped) and not the one that matters (is the answer actually good). That is the whole problem with agents on hard work in one finding. Getting to a feasible answer is the part agents are getting decent at. Getting to a dependable, correct, improved answer is the part that still needs a system around them.
Why this is an operating-model result, not a "wait for GPT-next" result#
It would be convenient if the fix were just the next model. The benchmark argues otherwise, and it lines up with the rest of the independent research. Apple's reasoning work found a complexity cliff where accuracy collapses on hard problems even when the model is handed the algorithm. Berkeley's MAST found multi-agent failures live in specification and coordination, not capability. ORAgentBench adds the end-to-end execution layer to the same story: the ceiling is the operating model around the agent.
Which means the work is the part you build:
Ground the agent in the real rules. Most "missed a constraint" failures are the agent not having the binding rule in front of it. Serve the actual spec and current data, not a vague summary. This is the one place we have a first-party number: trovex cuts roughly 60% of the tokens per lookup by serving the currently-correct slice for the task, so the agent works from the real constraints instead of a stale paraphrase.
Verify against feasibility, not vibes. A check the output can actually fail, run on the real constraints, before the work counts. ORAgentBench graded feasibility and quality separately for a reason; your pipeline should too.
Don't accept the first feasible draft. Build the improve-and-recheck loop the strong engineer does by reflex. The agent's first valid answer is a starting point, not a deliverable.
Worth keeping the claim precise, because the headline cuts both ways. A third pass rate isn't zero, and on bounded, well-specified tasks agents do considerably better than on these open, hard ones. The lesson isn't "agents can't work." It's that the gap between a demo and dependable end-to-end work is real, measured, and mostly made of operational discipline. Scope agents to where they're strong, build the system that supplies the discipline they lack, and you get the upside without betting your critical path on a 20% pass rate.
We run agent fleets to build and ship our own software, so we've met every one of these failure modes in production: the agent that ignored a constraint, the one that shipped a feasible-but-bad answer, the one that needed a gate it didn't have. The model is the part we change least. The grounding, the verification, and the improve loop are the work, and they're what turns a 20% agent into a dependable one for the slice of work you actually trust it with.
If you're putting agents on real work and the results are plausible but not dependable, that gap is the job. Talk to us about your setup.
We map where agents will help your codebase and where they'll quietly ship a feasible-but-wrong answer.
Can AI agents complete complex tasks on their own?
Not reliably, by current evidence. ORAgentBench's 2026 benchmark gave 14 frontier agent setups 107 real operations-research tasks; the best finished 35.5% overall and 20.6% of the hard ones. Many produced feasible-looking submissions that failed the quality bar. Agents handle bounded, well-specified work far better than open end-to-end tasks with real constraints.
Why do AI agents fail on hard end-to-end tasks?
The failures are procedural, not a reasoning deficit. ORAgentBench found errors dominated by missed operational rules, brittle problem formulations, and weak construction and improvement of feasible solutions. The agent produces plausible code that ignores a constraint or stops at a mediocre answer, which is a workflow and verification gap, not a model-IQ gap.
Will a smarter model fix agent reliability on complex work?
Only partly. ORAgentBench found that OR-specific procedural skills raised hard-task feasibility but did not reliably improve solution quality or pass rate. That points the fix at the operating layer, decomposition, verification against real constraints, and a quality gate, rather than at raw model capability alone.
How should teams deploy AI agents given these results?
Scope agents to bounded, well-specified work, ground them in the actual rules and data, and put a verification gate between their output and anything that ships. Treat an agent's first feasible answer as a draft to be checked and improved, not a finished result. The reliability comes from the system around the agent.
