verification

A good test suite is, first and foremost, a risk-mitigation measure.
Automated tests reduce the risk associated with changes to an existing codebase - most regressions and bugs are caught in the continuous integration pipeline and never reach users. The team is therefore empowered to iterate faster and release more often.

LLMs are not great at following commands. They deviate. Frequently. And they’re unreliable at self-verification—they’ll confidently tell you the code works while it’s on fire. The fix isn’t to ask the LLM to verify. It’s to ask it to write a script that verifies. Shift from judgment to artifact.

Instead of asking an LLM “Did this work?” you define verification steps that produce a series of pass/fail artifacts. The agent can’t negotiate with a failing test. It either meets the specification or it doesn’t.

We argue that coding agents have crossed a threshold of capability at which traditional human code review is no longer a necessary component of a software quality pipeline. Our argument rests on two claims: every stated goal of code review can be served by agents at lower cost and higher throughput; the naive integration in which agents write code and humans remain the mandatory reviewers is a dead end because it neither provides meaningful assurance nor scales with AI-assisted throughput.

Drawing on telemetry from over 10,000 developers across 1,255 teams, Faros’ recent landmark research report confirms:
Developers using AI are writing more code and completing more tasks
Developers using AI are parallelizing more workstreams
AI-augmented code is getting bigger and buggier, and shifting the bottleneck to review
Any correlation between AI adoption and key performance metrics evaporates at the company level

Extensive qualitative analysis of enterprise software engineers reveals that AI’s impact on the SDLC is not a simple linear improvement. Instead, it presents a series of profound tradeoffs. While AI successfully accelerates initial code generation and reduces the friction of starting new tasks, the time saved in creation is frequently re-allocated to auditing and verification. This tension may explain some of our own findings: higher AI adoption is associated with an increase in both software delivery throughput and software delivery instability.

The 2026 GenAI Code Security Report found that roughly 44% of AI code generation tasks introduced a risky security vulnerability in tests. The average security pass rate across models is 56% – barely changed from 55% in the first report. In other words, security performance has stayed flat while the amount of AI-generated code entering pipelines has surged.
That is why GenAI code security is now a scale problem, not a theoretical risk discussion. If AI is responsible for half the codebase and vulnerable output remains this common, every organization needs a sharper strategy for model selection, verification, remediation, and governance.

Reviewing AI generated code can be tough, especially if the code generated is solving a particularly complex problem. Even if the code is correct, it can be hard to understand how exactly it works. When working with other human programmers, I would generally ask them to walk me through their code, and I would ask questions about why they made certain decisions as we go. I wanted a similar process for reviewing AI generated code. After a lot of exploring, I've found a combination that works well for me.

Building the infrastructure to make verification routine is now a national-scale engineering problem.