20 June 20264 min read

AI helps most where you need it least

A Stanford study of 100,000+ developers found AI delivers big gains on clean, greenfield work and almost none on the complex, legacy code where most teams actually live. The lever isn't the model. It's the state of your codebase, and that's something you can change.

tsukumo

Short version: the AI productivity pitch is built on the easy case. New project, clean slate, simple feature. That's where the tools shine, and that's almost never where your team spends its day. Stanford went and measured the hard case, the messy existing codebase, and the gains mostly evaporate. Some teams come out behind. The uncomfortable read: AI pays off in proportion to how good your codebase already was.

What the Stanford study measured#

A team led by Yegor Denisov-Blanch at Stanford studied AI's real effect on developer productivity using private-repository data from over 100,000 developers across 600+ companies. Not a survey of how fast people felt. Actual code, measured. The split that matters:

Greenfield, low-complexity work: 35-40% gains. This is the demo, and the demo is honest, on this kind of work.
This is most real work.

High-complexity brownfield work: single digits, roughly 0-10%.

Some teams went net-negative, where the extra rework and debugging more than cancelled the faster typing.

Brownfield just means changing code that already exists, with its history, dependencies, and constraints. It's where mature software lives. And it's exactly where AI's advantage shrinks toward zero.

Greenfield versus brownfield: the gap is the story#

Why such a spread? Because a greenfield function has no past to respect. The model writes something reasonable and it's fine. A brownfield change has to fit decisions made three years ago by someone who left, around constraints that aren't written down anywhere the model can read.

So the agent produces code that looks right and doesn't fit. It misses the convention, duplicates a helper that already exists, breaks an assumption two modules over. The team then spends the time it "saved" putting that right. In one enterprise case in the Stanford work, AI adoption drove a 2.5x increase in code rework, which is another way of saying the speedup was mostly an illusion paid back later.

Telling the real gain from the illusion is exactly what an agent-ops assessment is for: measuring net productivity on your actual code, not the demo's.

Why clean codebases get more from AI#

The most useful finding isn't the gap. It's what predicts which side of it you land on. Stanford tied net productivity to an "Environment Cleanliness" measure: test coverage, type coverage, documentation, modularity, static code quality. The cleaner the environment, the more of AI's raw speed survived as real output.

That makes mechanical sense. Good tests catch the agent's mistakes before they cost you a day. Types and clear modules give it a target it can actually hit. Documentation is context it can use instead of guessing. The same investments that make a codebase pleasant for humans make it legible for an agent. AI doesn't replace engineering quality. It compounds it, in whichever direction it points.

This is good news, actually#

It's tempting to read all this as "AI doesn't work." That's the wrong lesson, and it lines up with the rest of the independent research: AI is a multiplier on your operating model, not a substitute for it. METR found the same perception gap; GitClear found the same quality drift. Stanford just located the dial.

And a dial is good news, because it's investable. You can't make the model dramatically smarter this quarter. You can raise your test coverage, tighten your modules, and fix your context. Those move the number AI multiplies. The team that does the boring work gets the exciting result.

What to do on Monday#

Stop benchmarking AI on greenfield. A clean-slate demo tells you almost nothing about your brownfield reality. Pilot it on the messy code you actually maintain.
Measure rework, not output. If AI-assisted changes get reverted or rewritten more often, you're in the net-negative zone, and that's a signal to fix the environment first.
Fund the cleanliness. Test coverage and modularity aren't separate from your AI strategy. They are your AI strategy. They set the ceiling on every gain.

How we think about it#

We run agent fleets in production on real, existing codebases, so we live in the brownfield case the demos skip. The thing that moves the number isn't a better prompt. It's the unglamorous work of giving the agent a clean environment and trusted context to operate in. This is also where our one hard number lives: trovex cuts roughly 60% of the tokens per lookup by serving the currently-correct code, so the agent fits the codebase instead of fighting it. The model is maybe 10% of a working setup. The other 90% is the environment Stanford just proved sets the ceiling.

If AI is underdelivering on your real codebase, that gap is the work. Talk to us about your setup.

Clean code is one of five levers: how to actually make AI work for your dev team.

Common questions

Does AI actually boost developer productivity?

Sometimes a lot, sometimes not at all, and the difference is the codebase. The Stanford study found large gains on new, simple code and gains near zero on complex, legacy code. Since most work happens in the second category, the average team sees far less than the demos promise.

What did the Stanford 100,000-developer study find?

Led by Yegor Denisov-Blanch, it analyzed private-repo data from over 100,000 developers at 600+ companies. AI gains ran 35-40% on greenfield/low-complexity work but dropped to roughly 0-10% on high-complexity brownfield work. In one enterprise case, AI adoption raised code rework 2.5x, cancelling the speed gain.

Why does AI help less on legacy or brownfield code?

Brownfield code carries history, constraints, and conventions the model can't see from a prompt. So it generates plausible code that doesn't fit, and the team spends the saved time on rework: fixing bugs, undoing hallucinated logic, and matching the existing style. The harder the code, the bigger that tax.

How do you get more from AI on an existing codebase?

Raise the codebase's cleanliness. Stanford found net productivity rises with test coverage, type coverage, documentation, modularity, and static quality. Better tests and structure give the agent a clearer target and catch its mistakes, so more of the speed survives as real output.

Want this running on your team?

Get your assessment