Surviving AI-Generated Pull Requests as a Reviewer

Last sprint I reviewed a pull request that was, by every surface measure, excellent. Clean commits. A tidy description. Tests that passed. Naming that matched our conventions better than most humans manage. It took me twenty minutes to notice it silently swallowed an error that we very much needed to propagate, in a code path that only fires during a partial outage — exactly when you least want a swallowed error.

The author was an engineer I trust, using an agent to draft the change. The agent wrote something that looked right, the tests didn't model the failure case, and the plausibility of everything around the bug is what nearly got it through. That's the new shape of the problem. Not bad code that looks bad. Correct-looking code that's subtly wrong, produced faster than anyone can carefully read it.

The bottleneck moved to review, and nobody staffed it

For most of my career, writing was the slow part and review was the fast part. You'd spend a day on a change and I'd spend twenty minutes confirming it. That ratio is now inverted for a lot of work. An engineer with an agent can open five substantial PRs in the time it used to take to open one, and each of them lands on someone's review queue expecting the same twenty minutes.

The math doesn't survive that. If generation gets 5x cheaper and review stays the same, review becomes the constraint on the whole system — and the natural, quiet response is to review less carefully. That's the failure mode I'm watching for on my own team. Not that AI writes bad code. That the flood of plausible code erodes the care we bring to each patch, because careful review of ten PRs a day is not sustainable and something has to give.

The thing that gives is usually attention on the boring middle of the diff — the part that isn't the headline change, where the swallowed error lives.

Plausibility is not correctness, and the gap widened

Human-written code used to carry signals. A confusing variable name, an odd structure, a comment that didn't match the code — these were tells that something was under-thought, and they'd draw your eye to where the bugs actually were. Sloppy code looked sloppy. The mess was a map.

Agent-generated code erased that map. It's uniformly clean. The naming is good everywhere, the structure is consistent everywhere, the comments are plausible everywhere — including the three lines where the logic is wrong. The visual cues I spent years training on now point nowhere, because the surface quality is decoupled from the correctness underneath.

So I've had to consciously retrain what I look at. The polish tells me nothing now. What tells me something:

• The seams. Error handling, edge cases, the empty-list and null paths, concurrency, anything touching a boundary between systems. Agents are strongest at the happy path they've seen a thousand times and weakest exactly where your system is specific.
• The tests, adversarially. Not "do they pass" but "what don't they cover." Agent-written tests tend to assert the behavior the agent just wrote, which means a wrong implementation gets a test that faithfully locks in the wrong behavior. Passing tests are now weaker evidence, not stronger.
• The assumptions. Does the change assume this list is small? That this call is idempotent? That this runs single-threaded? Those hold today and break at scale, and the agent had no way to know your production reality.

Make the author do the verification, not the reviewer

The instinct when quality drops is to review harder. That doesn't scale, and it also puts the accountability in the wrong place. The person who opened the PR is responsible for it whether or not an agent typed it — and "the agent wrote it" is not a defense I accept, the same way "I copied it from Stack Overflow" was never one.

What we changed on my team is the contract around what a PR has to arrive with. A change is not ready for my eyes until the author can answer, in the description, the questions I'd otherwise reverse-engineer:

• What failure modes did you consider, and how does the code handle them?
• Which test would fail if the core logic were wrong? Point to it.
• What did you verify by running it, versus assuming?

This isn't bureaucracy. It's moving the verification back to the person with the most context — the author, who knows the intent — instead of the reviewer, who's reconstructing it from a diff. If an engineer can't answer those questions about their own PR, they didn't review the agent's output before asking me to, and the PR goes back. The bar isn't "did you write this yourself." The bar is "do you understand this well enough to defend it."

A useful side effect: this pressure makes people use the agent better. If you know you'll have to explain the error handling, you prompt for it, you read it, you test it. The requirement to defend the change closes the loop that "looks good, ship it" leaves open.

Let machines check what machines are bad at

Some of this load belongs on tooling, not people. If agents are strong on the happy path and weak on the seams, then the checks I want in CI are the ones that hammer the seams — the work humans are worst at doing repeatedly and consistently.

Property-based tests that generate adversarial inputs instead of the three cases someone thought of. Mutation testing that tells you whether your tests would actually catch a bug, which directly attacks the "tests that lock in wrong behavior" problem. Fault injection in integration tests, so the partial-outage path that hid my swallowed error gets exercised on every run instead of never. Strict coverage on error branches specifically, not overall percentage.

None of this is new. What's new is the priority. When most code was human-written and carried its tells, these checks were nice-to-haves. Now that the tells are gone and the volume is up, automated adversarial testing is the layer that catches what tired human reviewers reading their tenth diff will miss. The machine writes the plausible code; a different machine should try hardest to break it.

The takeaway

AI didn't make code review optional — it made review the part of the pipeline that actually matters, and the part most under strain. The old signals are gone: polish no longer correlates with correctness, and passing tests no longer imply working code, because the same tool wrote both.

Adapt on three fronts. Retrain your own attention onto the seams where agents are weak instead of the surface where they shine. Push verification back onto the author, who owns the change regardless of what typed it. And invest the freed-up generation time into automated tests that attack the failure paths humans skim past. The volume isn't going down. The only question is whether your review process assumes the code arriving is trustworthy — because it isn't, and it never was, but now it looks like it is.