Something's off in how teams talk about their test suites.
The 2026 State of AI Code report revealed that 78.1% of engineering leaders trust AI-generated code more than they did a year ago. But in just the past quarter, 61% shipped a production incident that started in AI-generated code. Trust is climbing. But so is the mess left behind.
While the instinct might be to blame AI, looking at where incidents get through paints a different picture. When tests stay green right up until production, the problem isn't that AI writes bad code.
Testing infrastructure’ has meant the same thing for a decade: test environments, automation frameworks, a CI/CD pipeline to run it all, and a test management tool to track coverage. That definition is no longer sufficient but most teams haven't caught up.
The friction between trust in AI code and production incident frequency is a wake-up call: perception is lagging well behind what the infrastructure in place can actually catch.
Key takeaways
- Teams built half the AI loop but left the expensive half to humans. Every team surveyed uses AI to write code, but only 69.5% use AI to verify it works under real conditions, and only 48.6% use AI-generated unit tests.
- AI didn't save review time, it moved it. 64.8% of engineering leaders say AI-generated code needs more review time, and half report review cycles have grown 25% or more since adopting AI coding tools. 28.6% say senior engineers now spend more time reviewing code than building anything themselves.
- Flaky tests are the cost of the AI code adoption-verification gap. A 500-test suite costs roughly $360,750 a month to maintain manually, with selector changes (32%) and flow changes (27%) driving most failures, per Checksum's 2026 benchmark of 1M+ test runs.
The accepted definition of ‘testing infrastructure’
Ask vendors what testing infrastructure means and you'll get an answer like this: test environments that mirror production, an automation framework like Playwright or Cypress, a CI/CD pipeline that runs the suite on every change, and a test management tool to track what's covered and what isn't. That's the foundation nearly every engineering team needs, and most teams already have some version of it.
But that definition was built for a world where a human wrote most of the code running through it—at a human's pace. It answers questions like where tests run and how they get triggered. It doesn't answer whether the thing being tested was ever fully understood by whoever, or whatever, wrote it.
That gap didn't matter much when engineers wrote code line by line. It matters a lot now that they don't.
Why is testing infrastructure buckling now?
Trust in AI-generated code is climbing, and on paper, so is trust in the tests meant to catch bugs: 81.9% of engineering leaders say they trust their test suite more than they did a year ago, according to the State of AI Code report. But even while leaders are more confident in the AI and more confident in the safety net underneath it, 74.3% of them have watched that same safety net miss something anyway—in just the past 90 days.
The cost shows up in review, too. 64.8% of leaders say AI-generated code needs more review time than human-written code before they'll merge it, and half report their review cycles have grown by 25% or more since adopting AI coding tools. That tracks with what Faros AI found across its telemetry from more than 10,000 developers: teams with high AI adoption merge 98% more pull requests, and review time on those PRs is up 91%.
Whatever time AI saves on writing code, teams are spending back on reviewing it.
It's landing hardest on the most expensive employees; 28.6% of leaders say their most senior engineers now spend more time reviewing code, including AI output, than building anything themselves. One respondent described the dilemma: “Reviewing AI code takes longer because humans must meticulously hunt for subtle, confident hallucinations and edge-case bugs, turning peer reviews into an exhausting, high-cognitive-load debugging session rather than a standard logic check.”
That paints a picture of a team burning out compensating, by hand, for verification infrastructure that isn't doing its job.
The adoption-verification gap
Why hasn't testing infrastructure kept pace? The problem is that most teams built out one half of the AI loop and left the other half to humans. Every team surveyed for the State of AI Code report uses AI to write code but only 69.5% use AI to verify that code works under real conditions, and AI-generated unit tests (the most direct counterweight to AI-generated code) trail behind everything else at 48.6%, even behind AI-assisted code review at 61.9%.
Generation is the cheap half of the loop. Verification is the expensive half.
There’s a sense of familiarity for those who have been in the industry a while. “It's the same gap CI/CD left behind a decade ago,” the report notes. “CI/CD automated how code moves to production, not what that code does when it gets there.”
AI coding tools have done the same thing again, at a much larger scale: they've poured volume into one end of the pipeline without adding anything to verify what comes out the other end.
Current testing infrastructure was never built to close that gap. It was designed to run tests reliably, not to verify code nobody on the team actually wrote.
Introducing the Context Void
Ask engineering leaders what actually breaks and no single cause dominates. AI writes code without seeing the database it will query, the third-party API it will call, the load it will run under, or the business logic a developer holds in their head. That blindness doesn't produce one predictable kind of failure, which makes solving it impossible with yesterday’s testing infrastructure.
We call it the Context Void.
One respondent described what happens when a bug gets through, having functioned in unit testing only to fail in production: "Because the AI failed to take into consideration the rate constraints of a legacy inventory API, it produced a discount calculation function that functioned in unit testing but failed in production. This resulted in checkout timeouts lasting 20 minutes until we rolled back and manually rewrote the integration layer."
Asked what they'd fix with unlimited resources, many leaders alluded to testing infrastructure built for the AI age: “If I could wave a wand, I would fix the lack of true intent verification, meaning how we ensure AI code actually matches a developer's nuanced goals. Current tools check if code runs and passes basic tests, but they cannot verify if it aligns with complex, unwritten business logic.”
Flaky tests and test maintenance are symptoms of a wider issue
Most teams don't experience the Context Void as an abstract concept. They experience it as flaky tests: a suite that goes red for reasons that have nothing to do with a real regression.
Checksum's 2026 QA benchmark report, drawn from more than 1 million production test runs, calculates how expensive that can get. The median failure rate for manually maintained suites is 14.8 per 100 runs. A 500-test suite costs roughly $360,750 a month to maintain without AI assistance, or $4.3M a year.
The breakdown of why tests break tells its own story: selector changes account for 32% of failures, flow changes 27%, environment instability 22%, and loading or timing issues 19%. None of that is a real regression. It's a test that fell out of sync with an application that kept moving.
Flaky tests and endless test maintenance aren't a separate problem from the adoption-verification gap. They're what that gap looks like day to day, on a team still trying to run legacy testing infrastructure against an application that changes faster than a human can keep up with by hand. (For a closer look at why tests go flaky and what to do about it, see Checksum's breakdown of flaky test causes.)
What does testing infrastructure look like for AI code?
If the old definition of testing infrastructure was environments, tools, and pipelines, the new definition adds four properties layered on top:
- Continuous: Tests run on every change instead of waiting for someone to remember to write or trigger them
- Production-like: Verification happens against real data and real APIs instead of a clean unit-test fixture
- Self-maintaining: UI changes or a shifted flow is auto-healed instead of sitting broken until an engineer has time
- Owned: Teams aren’t locked into a proprietary format they can't take with them
Notably, none of that requires standing up a second system. The infrastructure most teams already run (e.g. GitHub Actions, CircleCI, or Jenkins for CI/CD, GitHub or GitLab for source control, Playwright or Cypress for the tests themselves) doesn't need to be replaced. The existing system needs a different layer running inside it.
This is what continuous, auto-healing end-to-end coverage looks like. Checksum's E2E Agent detects the critical user journeys in an application and generates standard Playwright tests for them. These are delivered as PRs—never a direct commit—so an engineer reviews exactly what changed before it merges. When a selector or a flow shifts, the same agent opens a healing PR instead of leaving a red test for someone to chase down by hand.
Across Checksum's benchmark data, around 70% of failures resolve fully autonomously this way, and with a human reviewing the fix, about 98% resolve in under ten minutes.
What good looks like
Teams adopting Checksum haven’t built new testing infrastructure from scratch; they’ve changed what runs inside the infrastructure they already have.
Postilize, a legal tech SaaS company, was stuck in constant regression firefighting that broke momentum after every release. Closing the gap got them 70% fewer bugs, 30% faster engineering cycles, and 0% flakiness in their suite.
ClearPoint Strategy was losing too many bugs to customers and too much engineering time to test maintenance. The fix: $500,000 in annual savings and six critical bugs caught weekly, before they shipped.
Counterpart, an insurtech company with growing product complexity and no dedicated QA headcount, now catches one significant production issue a week with zero production outages, at roughly 10x the impact of a QA team they never had to hire.
The old definition of testing infrastructure is obsolete
Environments, frameworks, and CI/CD pipelines are still the foundation every team needs. What's obsolete is treating that foundation as sufficient on its own, now that most of the code moving through it wasn’t written by the person reviewing it.
Checksum's E2E Agent and API Agent are built to be the layer that verifies AI code: continuous instead of on-demand, verified against production-like conditions instead of clean fixtures, auto-healing through generated PRs instead of manual triage, and delivered as standard Playwright or pytest code a team owns outright. CI/CD automated how code gets delivered. Continuous verification automates how it gets proven.
Want the full picture of what's breaking in AI-generated code and why? Read Checksum's 2026 QA benchmark report, or see how the E2E Agent works.
FAQs
What is the "Context Void" in AI-generated code?
The Context Void is the blind spot AI coding tools have when they generate code: they can't see the database it will query, the third-party APIs it will call, the load it will run under, or the business logic a developer holds in their head. That blindness is why code can pass unit tests and still fail in production.
Why does AI-generated code need different testing infrastructure than human-written code
Traditional testing infrastructure (e.g. environments, automation frameworks, CI/CD pipelines, test management tools) was built to run tests reliably for code a human wrote at a human's pace. It confirms tests run and get triggered, but it doesn't verify whether AI-generated code matches a developer's intent or holds up under production-like conditions.
What makes testing infrastructure ready for AI-generated code?
Checksum defines AI-ready testing infrastructure by four properties layered on top of the existing foundation: continuous (tests run on every change, not just when triggered), production-like (verified against real data and APIs instead of clean fixtures), self-maintaining (auto-healed instead of left broken for an engineer to fix), and owned (delivered as standard code, not a proprietary format).
