The Naive Baseline Fallacy

There is a methodological flaw embedded in most published research on agentic AI for software development, and it runs deep enough that it has shaped the entire public discourse. Studies deploy AI tools under poor prompting conditions, with no invariant formalization, no architectural enforcement, no adversarial test generation, and no multi-agent critique loops. They measure the results. They publish those results as characteristic AI performance. And the industry press — which selects for clean, alarming numbers — amplifies them into the conventional wisdom.

The conventional wisdom is wrong. Or rather: it's measuring the wrong thing and drawing the wrong conclusion from it.

Call this the naive baseline fallacy: the error of evaluating AI under unengineered default conditions and then treating those results as if they reveal something about AI's fundamental ceiling rather than something about the floor of engineering practice around it.

Two Very Different Experiments

To understand what's being missed, it helps to name the two evaluative frames clearly — because most published studies conflate them.

Frame A asks: "How does the model perform under typical use?" It produces the Copilot productivity studies, the vulnerability injection rate reports, the benchmark coding tasks. These are real measurements. They're also measurements of a specific, narrow thing: AI used by engineers who haven't adapted their practice to AI's particular strengths and failure modes.

Frame B asks something categorically different: "What happens when engineering practice is redesigned around AI?" What happens when invariants are formally declared before code generation begins? When architecture is machine-enforced in continuous integration? When test generation is adversarially prompted? When agents critique each other? When PRs require explanation-backed reasoning chains?

Frame B experiments are harder to run. They require real engineering redesign rather than a controlled trial with a static prompt. They take months, not weeks. And they produce different results — results that more accurately reflect what AI-augmented teams are actually capable of when they adapt.

The reason Frame B is underrepresented in the literature isn't that it's unimportant. It's that Frame A studies are faster to run, cheaper to replicate, and generate more press-ready headlines.

How Failure Narratives Travel

There's a propagation asymmetry that compounds the methodological problem, and it has nothing to do with bad faith.

"AI introduces security vulnerabilities at X% rate" is a clean, alarming, publishable finding. It moves from research paper to security conference proceedings to tech press to executive briefings to board risk reviews with minimal friction. The alarm is legible, the metric is concrete, and the call to action — be cautious — is easy to operationalize.

"Engineering teams that formalized invariant declarations before AI code generation saw a substantial reduction in semantic errors over ninety days" is harder to headline. It requires organizational context to interpret. It doesn't slot cleanly into existing risk frameworks. It requires the reader to understand what invariant formalization means and why it changes the outcome. The story is real — it's happening in practice — but it travels slowly.

The result is a systematic skew in the discourse. Failure studies accumulate. Adaptation stories — the rarer, harder, more consequential experiments — get less surface area. And organizations making investment decisions receive a distorted signal: that AI is inherently risky, that adoption should wait, that the technology isn't ready.

This has real costs. Delayed adoption means delayed access to the productivity compound. Misallocated risk management means teams build defenses against the wrong failure modes. Engineering investment goes toward monitoring and mitigation rather than formalization and co-design. The deterrence effect of failure framing isn't neutral — it shapes what gets built.

The Forcing Function Insight

The most consequential thing that failure-focused framing misses isn't a gap in results. It's a causal inversion.

When AI underperforms in a software engineering context, it is almost always because something in the engineering system is implicit. Requirements are ambiguous. Architecture is informal. Invariants exist in senior engineers' heads rather than in code. Test coverage is driven by convention rather than specification.

AI doesn't create these conditions. It exposes them.

When an AI agent generates code that violates architectural conventions the team hasn't formally encoded, the violation is visible precisely because AI lacks the contextual judgment to paper over it the way an experienced engineer would. When an AI test suite misses an edge case that human reviewers have always implicitly checked for, the gap is made legible in a way it wasn't before.

This is not degradation. It is structural maturation. And it means that teams measuring AI purely against short-horizon output quality metrics are missing the most significant long-term benefit: AI as a forcing function for engineering discipline.

The paradox this creates for published research is real. A study that captures the first six weeks of an AI deployment will find higher review time, more explicit formalization work, more engineering overhead. A study that captures the same team at six months will find higher invariant coverage, better architectural consistency, lower defect rates, and faster onboarding. These are measurements of the same technology in the same organization. They are not measurements of the same thing.

What 2026 Is Actually Showing

The shift is observable in practice patterns even if it hasn't yet fully penetrated the research literature. Teams that have been working with agentic AI for eighteen months or longer — with the benefit of learning its failure modes directly — are not using it the way it was used in the 2023 and 2024 studies that dominate the narrative.

They are using prompt pipelines instead of single prompts. They are extracting invariants before code generation begins. They are running agent self-critique loops as a standard workflow step rather than as an experimental addition. They are generating adversarial tests as part of the commit process. They are enforcing architecture as code, verified in CI. They are leveraging large context windows for repository-wide coherence checks that were impossible at smaller context sizes.

These are architectural changes to how human-AI collaboration is structured, not incremental improvements to how a tool is used. The quality ceiling of this system is substantially higher than what Frame A studies measure — and the divergence between teams operating this way and teams that aren't is widening.

This divergence is the story that should be getting more coverage. It is not yet.

The Metrics That Actually Matter

Part of why Frame B studies are rare is that they require different — and harder — measurements. The standard AI evaluation metrics are calibrated for Frame A, and they don't capture what matters in a co-designed system.

These metrics are harder to collect. They require longer study horizons. They require pre-AI baselines that many teams didn't establish. They require teams to have already made the adaptation investments whose effects are being measured. None of this is an argument against measuring them — it's an explanation of why they're underrepresented, and a case for fixing that.

The Security Case: Where the Fallacy Bites Hardest

Security is where the naive baseline fallacy does the most damage — not because the research is worse than in other domains, but because its audience is larger and its institutional leverage is greater. A study showing AI introduces SQL injection at measurable rates lands differently than a study about code review latency. It feeds directly into board-level risk assessments, AppSec program justifications, procurement security questionnaires, and CISO briefings. The deterrence effect runs deep and fast.

The findings themselves are not wrong. Under naive deployment conditions — default prompting, no security context, no toolchain integration — AI-generated code does introduce CWE Top-25 vulnerabilities: SQL injection, path traversal, insecure deserialization, buffer overflows, hardcoded credentials. Some published studies have found security weaknesses in a substantial fraction of AI-generated code samples when evaluated under these conditions. That measurement is accurate. The problem is the inference drawn from it.

When none of these are present, the measured vulnerability rate is a property of the deployment configuration, not of AI's security ceiling. The structural defenses are well understood. They are, in many cases, the same defenses that secure human-written code — the difference is that human senior engineers carry security intuitions implicitly, while AI requires those intuitions to be made explicit in the toolchain. That's not a fundamental limit. It's a formalization requirement.

The forcing function dynamic applies here with particular clarity. Human developers with strong security instincts paper over gaps that AI makes visible. A senior engineer never writes raw string concatenation in a SQL query because decades of security culture have made the danger instinctive. AI, without that history encoded, generates the construction unless forbidden. The gap is now explicit. When teams respond by encoding the prohibition formally — in prompt-level invariants, in SAST rules, in pre-commit hooks — they end up with a more auditable, more consistently enforced security posture than they had before. The AI didn't weaken their security. It revealed where it was relying on invisible expertise.

The organizational consequence of stopping at the vulnerability injection rate — without completing the analysis — is that security functions treat AI as an inherent liability rather than as a tool whose security properties are highly dependent on deployment architecture. AppSec programs built on Frame A findings end up in an adversarial posture toward AI adoption: monitoring for AI-generated code, flagging it for extra review, treating it as a category of risk rather than a class of tooling whose security profile can be engineered. That's not wrong given the studies they're drawing from. It's the wrong response to the right observation applied to the wrong frame.

The Frame B security question is not "does AI generate vulnerable code?" It is: "what is the residual vulnerability rate of AI-generated code when security invariants, SAST, adversarial security testing, and dependency enforcement are all active?" That study, run rigorously, with pre-AI baselines for comparison, would produce a much more useful input to enterprise AI policy than the studies currently driving it.

A Historical Parallel Worth Taking Seriously

Early compilers were compared to hand-written assembly code. The comparison was not flattering for compilers. Expert hand-optimized assembly consistently outperformed compiler output on benchmarks that mattered: execution speed, memory efficiency, instruction count. The conclusion a reasonable observer might have drawn: compilers are inferior. Use them for prototyping, perhaps, but not for production systems where performance matters.

That conclusion would have been precisely, catastrophically wrong — not because it was inaccurate about what it measured, but because it was measuring the wrong question.

The right question wasn't "is compiler output better than expert assembly?" It was "what becomes possible when engineers can think at the level of high-level abstractions rather than machine instructions?" The answer was abstraction, modularity, systems of previously unimaginable scale, entire new categories of software. The performance gap between compiler output and hand-optimized assembly turned out to be largely irrelevant to what actually mattered about the technology.

We are in the same position with AI. The question "is AI-generated code as reliable as carefully reviewed human code?" is the Frame A question. It has a Frame A answer that depends heavily on how the AI is deployed. The Frame B question is: "what becomes possible when AI is integrated into the development process as a designed cognitive system — with invariant enforcement, adversarial testing, multi-agent critique, and architectural formalization as standard components?"

That question doesn't have a full answer yet. But the research agenda that could answer it looks nothing like the studies currently dominating the discourse.

Structured Optimism

None of this is an argument for naive optimism about what current AI systems can do. The real limits are real. Long-horizon architectural coherence remains genuinely difficult. Production-scale intuition — the judgment accumulated from watching systems fail in production over years — is not something current models replicate. Emergent distributed behavior modeling is an unsolved problem. Causal simulation at the depth a senior engineer applies remains beyond current systems.

The argument is for structured optimism: recognizing that AI performance is highly elastic with respect to engineering system design, that the right response to identified weaknesses is engineering adaptation rather than adoption deterrence, and that the research community is systematically underinvesting in studying what that adaptation produces.

The bias toward failure framing isn't uniquely a failure of journalism or research culture. It is so structural that it surfaces even in analyses that are explicitly trying to think carefully about AI's role in software development — including AI-assisted analyses of this very question. The default frame, when confronted with the question "what are the risks of agentic AI?", is to enumerate failure modes under naive conditions. The follow-up question — "what engineering responses reduce those risks toward zero?" — requires an extra step of reasoning that the default frame doesn't supply. Recognizing this helps explain why good-faith coverage produces misleading conclusions: it's not that the individual studies are wrong about what they measure. It's that the accumulation of Frame A results, in the absence of Frame B results, produces a systematically incomplete picture.

The most important research question in this space isn't "does AI introduce bugs?" It does, under poor conditions. That's established. The question is: "what engineering structures maximize the quality ceiling of AI-augmented teams?" That is a design question, not a measurement question. And design questions require experiments that look more like engineering than like controlled trials.

The teams running those experiments — quietly, in production, without publishing papers — are already several moves ahead. The research literature will catch up. The press narrative, which depends on the research literature, will take longer. In the meantime, organizations making AI investment decisions should be skeptical of failure framings that stop at the floor and mistake it for the ceiling.