Post
JA EN

Checking Can't Keep Up: Stop Processing the Volume AI Created, Invest in Quality and Findability

Checking Can't Keep Up: Stop Processing the Volume AI Created, Invest in Quality and Findability
  • Target audience: Managers whose review load grew after AI arrived, tech leads and EMs designing review workflows, and anyone on a team handling large volumes of AI output
  • Prerequisites: Some hands-on use of generative AI at work
  • Reading time: about 21 minutes

Overview

AI joined the team and the output went up. More pull requests, more proposal drafts, more design notes, more research write-ups. Individually, everyone really is faster. And yet the pile of things a manager has to look at keeps growing, and the time available per item keeps shrinking.

Leave that alone and the ending is predictable. Checking gets sloppy, defects that slipped past a sloppy check come back as rework, and the rework adds more things to check. The 2024 DORA report estimated that a 25% increase in AI adoption came with a 7.2% decrease in delivery stability1. Individual productivity and satisfaction were up at the same time. By the 2025 report, AI had picked up a positive relationship with throughput, but the correlation with instability was still there. DORA’s explanation is that AI accelerates development, and the acceleration exposes weaknesses downstream2. Exactly where review sits in that downstream is the question a paper discussed later takes on directly3.

Which is why “how do we process all this extra volume” is probably the wrong question. You can raise the rate at which you process it, but you will not out-run the rate at which it is generated upstream.

This article proposes two investments, and neither of them makes checking faster. The first is to decide quality before volume, where quality explicitly includes being easy to review. The second is to make sure good work stays somewhere it can be found and reused immediately. Run both at once and the number of things that have to be checked starts falling. The same mistake stops recurring, and the same investigation stops being done twice.

DORA’s AI Capabilities Model, published in 2025, lists both “working in small batches” and “AI-accessible internal data” among the factors that amplify AI’s effects4. The point DORA keeps making is that AI is an amplifier. Organizations whose substance is in order get a multiplier; organizations whose substance isn’t get their weak spots broken sooner. Both directions in this article work on the substance side, before the amplification.

What follows: why checking gets heavier faster than volume grows, what the two directions actually consist of, and how they relate to the other ways of reducing review load (automated gates, risk-based sampling, AI reviewers, more reviewers).

1. “Review hell” has been measured

Worth establishing first that this isn’t just a feeling.

In a survey of 1,150 US full-time desk workers by BetterUp Labs and Stanford Social Media Lab, 40% of respondents said they had received “workslop” in the previous month5. Workslop is their coinage for AI-generated output that looks polished but lacks depth, accuracy, or context. Dealing with one instance took an average of 1 hour 56 minutes, which works back to about $186 per person per month from self-reported salaries. At a 10,000-person company that clears $9 million a year.

The part that matters in that number is that the cost lands on the receiving side. The producer’s time really was saved. The saving moved downstream and became someone else’s checking cost. (This survey also came up in Stop Chasing AI News, Build Workflows.)

In engineering the effect shows up more directly. Stack Overflow’s 2025 developer survey (over 49,000 respondents across 177 countries) found 66% frustrated by AI solutions that are “almost right but not quite,” and 45% saying debugging AI-generated code eats their time6. Trust in the accuracy of AI output sat at 33% (only 3% “highly trust”), against 46% who don’t trust it.

“Almost right but not quite” has the worst possible property from a checking standpoint. Something obviously broken gets thrown out immediately. Something 80% correct forces you to read the whole thing to find the other 20%. The reading volume is unchanged; only the density of findings drops.

On the empirical side, AI-authored PRs have been reported to get reviewed less often, merged several times faster, and discussed less3. The paper itself cautions that this tendency is unstable and can reverse under different analytical choices. Its central claim carries more weight here. Review is the control point that determines the sign of a coding agent’s effect on software, and AI does not set that sign. The team does.

2. Doubling the volume does not double the checking

Why does checking fall behind? If double the output meant double the review load, hiring would still be an answer. It doesn’t work that way, for three reasons that stack.

2.1 Queues grow non-linearly with utilization

As has been pointed out repeatedly in the product development flow literature, queue length does not grow linearly with utilization. Going from 60% to 80% utilization roughly doubles the queue, and it climbs far more steeply as you approach 95%7.

A reviewer’s time is a fixed capacity, and review requests are jobs arriving at it. Say your reviewers spent 60% of the week on review before AI, and generation volume pushed that to 80%. Wait time isn’t the only thing that stretches. Context goes stale while work sits, conflicts multiply, and the author has moved on to something else, so responses to comments come back slower too. Secondary load rides on top of the queue length.

That said, the queueing model assumes random arrivals and no interruptions, which is not how human review works. Priorities let jobs jump the line, and reviewers are running other work in parallel. What I want from the model is not the accuracy of the multiplier but the shape: load does not translate into pain linearly.

2.2 Bigger changes, lower detection

SmartBear’s study at Cisco Systems, covering 2,500 reviews and 3.2 million lines over ten months, concluded that the workable size for a single review is 200 to 400 lines, and that the ability to find defects drops past that8. Above roughly 500 lines per hour, defect density falls off clearly, and after 60 continuous minutes reviewers are spent and stop finding anything more.

Google’s case study across 9 million changes gives a contrasting set of numbers. Over 35% of changes modify a single file, and over 75% have one reviewer9. Small changes get first feedback in under an hour at the median; very large ones take about five. The overall median review time is under four hours. They’re not fast because the changes happen to be small. They’re fast because the changes are kept small.

The trouble is that AI is not good at producing small changes. Ask it and it will emit a large change in one shot, and that looks efficient. The 2024 DORA report also raised growing batch size as a hypothesis for why AI reduces stability1.

2.3 “Probably fine” accumulates

The third reason follows from the first two. There isn’t enough time, each item is large, and the defect density is known to be low. What a human does under those conditions is sample without noticing.

Sampling deliberately, on the basis of risk, is a legitimate strategy, as covered later. The problem is arriving there through fatigue and running out of time, with no criteria. Nothing records what was looked at and what wasn’t, so when something slips through, the cause is unrecoverable.

flowchart TB
    A["AI raises output volume"] --> B["The review queue grows"]
    B --> C["Each check gets shallower"]
    C --> D["Defects and mismatched<br>assumptions flow downstream"]
    D --> E["Rework and re-explanation"]
    E --> F["More things to check"]
    F --> B

3. Direction one: decide quality before volume

“Prioritize quality” here is not an appeal to care more. It’s a design decision: put the properties that lower the reviewer’s cost into the requirements for the work itself.

3.1 Put reviewability inside the definition of quality

In most organizations “high quality” means something like no bugs, meets the spec, readable. Add three things to that.

One, it is small. One change or one document carries one intent. SmartBear’s 200 to 400 lines and Google’s single-file norm are both institutional implementations of that property. DORA’s AI Capabilities Model likewise reports that working in small batches amplifies AI’s positive effect on product performance4.

Two, the intent is written down first. What changed is readable from the diff. Why it was decided that way, which alternatives were dropped, what has been verified and what hasn’t: none of that is recoverable unless someone wrote it. Most of the cost of checking goes into guessing at exactly this. It’s worse in AI-produced work, because no human judgment entered the generation, so if it isn’t written down it genuinely doesn’t exist anywhere.

Three, it verifies itself. Tests pass, links resolve, numbers match their sources. Any time a human spends confirming something a machine could confirm is time you can delete outright.

Satisfy those three and the reviewer’s job changes from hunting for mistakes to assessing whether the judgment was sound. The first scales with volume. The second doesn’t, or at least not nearly as steeply.

3.2 A human passes over it before it ships

Push AI output straight into the next stage and the total amount of checking doesn’t drop. Someone will read it regardless. The only variable is whether that someone is the author or the reviewer.

The author is cheaper. They have the context, and fixing what’s wrong is faster for them. The reason workslop’s cost landed on the receiving side is that this one step was skipped.

The point isn’t to stop using AI. It’s to make the split explicit at the level of each deliverable: AI generates, a human is accountable. Work where nobody has written “AI produced this, I checked it, and these parts are my call” reads, from the reviewer’s chair, as a document with an unknown starting point.

3.3 Don’t spend the freed time on more work

This is the hard part.

When AI makes work faster, the freed time has three possible destinations. Take on more work, spend it on raising quality, or leave it unfilled. The 2024 DORA result where individual productivity, flow, and job satisfaction all rose (by 2.1%, 2.6%, and 2.2% respectively) while organizational throughput fell1 suggests the freed time was not converting into organizational output. Take the first option without noticing and utilization rises, which just lengthens the queue from §2.1.

The second option, spending it on quality, is quite defensible in this context. Study design, understand the domain you work in more deeply, get better at instructing AI, tidy up the systems you already have. All of that raises the quality of subsequent work and lowers the cost of checking it. And the effect compounds. Finishing one item faster ends with that item; raising the quality bar by a notch applies to everything after it.

The third option, leaving it unfilled, is less silly than it sounds. From the queueing argument, 100% utilization is the worst possible setting. With no slack to absorb interruptions and incidents, a single problem sends everyone’s wait times up.

But left alone it will always become the first option. Work expands to fill the time available, and time that looks free attracts other work. Learning and maintenance time disappears unless a manager explicitly says this block does not get filled. Team-level good intentions can’t protect it. It has to be handled as a decision.

3.4 A quality bar nobody wrote down does nothing

“Let’s raise quality” changes nothing on its own, because people mean different things by it.

The minimum you have to write down, per PR or per document, is roughly this. What has to be present for it to be accepted (intent, rejected alternatives, what was verified, what wasn’t). How much must be machine-verified before it arrives. What the size ceiling is for one item. And, going the other way, what does not need to be written.

That last one carries a lot of weight. Left unspecified, an instruction to raise quality gets read as a demand for completeness and produces enormous templates. Which is the workslop factory from §4.3.

3.5 A starting bar, for argument’s sake

To avoid leaving this abstract, here are concrete numbers for code changes. Not to adopt verbatim, but as something to argue with.

  • One PR stays under 200 lines as a rule, 400 at the absolute most. If it exceeds 200, put one line in the description explaining why it can’t be split
  • The description covers four items: what the change is for, alternatives considered and rejected, what was verified, what was not verified
  • Lint, types, tests, and link checks pass in CI before the review request goes out
  • For anything AI generated, add one line on which parts were your own call
  • Nothing beyond that. No long design essays, no restating changes the diff already shows, no test-running instructions in the description

The last line is the “what not to write” from §3.4. Without it, the four-item list gets read as a completeness requirement and the prose piles up to satisfy it.

For documents, the line budget becomes “one file per decision,” and the verification scope becomes “where does this claim come from.”

The 200 and 400 come straight from the study cited in §2.2 (its own phrasing is to stay under 200 lines and never exceed 400). Move them for your domain’s risk profile, and sometimes you should. What actually helps is the argument that happens when you move them: what is the right number for us? With no number written down, that argument never starts.

One thing worth trying before setting the bar: count how many lines your team’s last ten PRs actually were. The gap between the felt number and the real one is common enough.

4. Direction two: make knowledge findable and immediately usable

You can produce excellent work, but if nobody ever refers to it again, the next person gets stuck in the same place, asks the same question, and receives the same review comment. The volume to check doesn’t fall.

This section covers part of a larger topic, an organization’s capacity to supply context, which is laid out in full in Build Your Organization’s Context Supply Capability First. Here I’ll stay on what changed once AI arrived.

4.1 Searching was already expensive

A 2012 McKinsey Global Institute report estimated that interaction workers spend about 20% of the week searching for internal information or hunting for a colleague who can help10. The same report suggested that making internal exchanges searchable could cut that search time by up to 35%. It’s a 2012 estimate over a broad population, so it isn’t a number to apply directly to your own organization. The order of magnitude is still hard to dismiss.

That time doesn’t disappear when AI arrives. It changes shape. Instead of a person searching internally, the person asks AI, AI doesn’t know the internal context and returns something generic, and the person adjusts it to fit. The adjusting is done by a human, and whether they can adjust it depends on whether they know the internal context. The search didn’t go away.

4.2 “Findable by AI” became a new requirement

Of the seven capabilities in DORA’s AI Capabilities Model, two concern the accessibility of information4. “Healthy data ecosystems” holds that high-quality, accessible, integrated internal data substantially amplifies AI adoption’s positive effect on organizational performance. “AI-accessible internal data” holds that connecting AI tools to internal data sources increases the effect on individual effectiveness and code quality.

So the state of internal knowledge is now a question about human productivity and, at the same time, a question about the inputs that determine AI output quality. This is where the two directions meet. If good work stays findable, it becomes the input to the next AI output, the next output is better, and checking gets lighter.

What to do concretely varies a lot by organization. A few things tend to work generally.

Start by picking one canonical place per theme. This is not about standardizing the whole company on one tool. Tool consolidation usually dies against migration cost and team-specific needs (that argument is developed in Tool Sprawl: When Single Source of Truth Collapses). What you decide is where the truth lives per theme: deployment procedures here, architecture decisions there. When the same information is scattered across four tools, neither AI nor a person can tell which copy is current.

Record decisions. Use the ADR (Architecture Decision Record) form: what was decided, why, and which options were dropped. Fix one decision per file, with a serial number and a fixed location like docs/adr/NNNN-short-title.md, and both humans and AI arrive without searching. The format reads well for people and lands well when handed to a model.

Then, expiry and ownership. Put a last-reviewed date and an owner at the top of the file, and move anything past, say, six months since last review onto a different shelf. Unlike the line counts in §3.5, the six months has no research behind it and is purely a starting number to be adjusted. What matters more than the interval is deciding whose inbox the notification lands in when it expires. When stale material sits on the same shelf as current material, search precision falls in inverse proportion to how much you have.

4.3 But adding volume backfires

A caution here: read “knowledge sharing” as “write more documents” and this fails reliably.

The 2024 DORA report estimated that a 25% increase in AI adoption came with a 7.5% improvement in documentation quality, the largest gain among the effects it estimated1. That does not mean using AI makes documentation better. In the same period we also have the finding that thin AI-mass-produced documents become a cost for whoever receives them (the workslop of §1).

The dividing line is whether human judgment is in the document. A document where AI re-summarized existing information adds no information and pollutes search results. Only documents carrying what was decided, what was rejected, what the constraints were, and what failed, the things only someone who was there can supply, help the next person (and the next model). The structure where writing is rewarded but retiring stale material isn’t, so dead documents accumulate, was covered in Documentation Theater.

5. What happens when both run together

Either direction alone gets limited results.

Raise quality without sharing and the quality stays locked inside individuals. It leaves when they leave, and the team next door steps on the same rake independently. Share without raising quality and you’ve made poor material searchable, which increases search time.

With both running, the loop reverses direction.

flowchart TB
    A["Agree the quality bar first<br>(small, intent stated, self-verified)"] --> B["Checking finishes faster"]
    B --> C["Freed time goes to<br>learning and upkeep"]
    C --> D["Work with judgment in it<br>stays findable"]
    D --> E["It becomes the next input<br>for people and AI"]
    E --> F["Good output on<br>the first pass"]
    F --> G["Fewer things to check"]
    G --> C

The one transition in that diagram that never happens on its own is B to C. Per §3.3, freed time fills with the next job if nobody stops it. That link has to be held by decision, and without it the loop stops at B and nothing else follows.

What’s shrinking here is not the time per check. It’s the number of checks. Settled questions stop reopening. The same investigation isn’t run twice. The mistake that got flagged last time stops arriving again.

There’s no quick win in this. It’s the kind of investment that pays over months, which makes it awkward as a quarterly objective. That seems worth stating plainly.

6. How this compares to the alternatives

Other approaches to review load exist and have track records. This article’s proposal doesn’t replace them, so it’s worth laying out the relationships.

6.1 Automated gates (give the machine what a machine can verify)

Put tests, static analysis, type checking, linters, and security scans in CI, and don’t send anything to human review until it passes. The idea is to convert claims a reviewer would otherwise have to take on faith into machine-checkable evidence.

The effect is large, implementation is comparatively easy, and it’s the first place to work when AI-generated code volume rises. It has the same shape as jidoka and poka-yoke in manufacturing, which tried to build quality into the process rather than inspect afterward, and thereby remove the inspection step.

The limit is that only machine-verifiable things get verified. Whether a design decision makes sense for the domain, or whether a spec reading matches customer intent, cannot be automated. And that is exactly the part AI output tends to get wrong.

Rather than a complement, automated gates are the institutional implementation of “it verifies itself” from direction one. The same idea approached from the design side, building things simply enough that checking becomes unnecessary, is covered in Simplicity Is the Fastest Strategy.

6.2 Risk-based review and sampling

Stop looking at every change at the same depth. Scrutinize authentication and billing; let a well-tested utility function through on automated gates without waiting for human approval. Some teams pair this with post-merge review, sampling merged changes after the fact.

This turns the unconscious sampling from §2.3 into a deliberate design. Criteria get written down, and what was looked at and what wasn’t gets recorded. It works well.

Two limits. Judging risk itself requires judgment. And quality can degrade gradually in the areas that fall outside the sample without anyone noticing.

This composes too, and the further direction one has progressed, the easier the risk call becomes. When one change carries one intent, whether it touches a high-risk area is close to mechanically decidable.

6.3 AI reviewers

Have AI do the review and reduce human load. A straightforward response to rising generation volume: scale the review side too.

But in a study analyzing 240 PRs and 739 review comments, AI (GPT-4) caught only about one-tenth of the quality problems humans find11. The same study reports that AI produced 2.4 times as many comments as humans, and roughly 40% of the extra ones were meaningful problems. So it doesn’t substitute for a human, but it does fill in the side a human isn’t looking at. Review articles land similarly: usable for coding conventions and minor defects, weak at detecting high-severity vulnerabilities that require deep data-flow analysis or security knowledge12. And the structure of a human checking AI output is unchanged, so review comments become a new category of thing to check.

You can combine it, but make it the main strategy and the structure from §1 reappears one level down. AI reviewers probably work best picking up mechanically decidable convention violations and oversights so humans can concentrate on judgment. (This is treated in more detail in Is “AI Does It, So Checking Can Be Light” True?.)

6.4 More reviewers, more review time

The most direct answer, and it works in the short run. Lower utilization and the queue shrinks (§2.1).

The limit is that unless the upstream generation rate drops, the added capacity fills up quickly. And every person moved into reviewing is a person no longer producing, so the organization as a whole hasn’t gained anything. Per §2.2, the review volume one person can absorb also has a ceiling from the detection-rate side.

Reasonable as an emergency measure, not as a permanent one.

6.5 Where each one bets

These don’t compete. They just bet in different places.

Automated gates lower the unit cost of checking, and to the extent they build quality into the process and eliminate rework, they also reduce the count. That was the point of jidoka. Risk-based sampling lowers the unit cost. AI reviewers and extra reviewers raise checking capacity. The two directions in this article bet squarely on the count.

Handle this with unit cost and capacity alone and, as long as generation volume keeps rising, something eventually catches up with you. Investing in reducing the count is the most direct way at that.

7. What you’ll run into

Rather than only describing the clean version, here are the things that will definitely be a problem.

“Raise quality” is a slogan without a bar. Per §3.4, say it without having written down what high quality means and the receiving side either reads it as “try harder” or converts it into filling out an exhaustive template. Both increase review load. Write the bar first.

The structural reason knowledge sharing doesn’t stick is an asymmetry: the cost falls on the writer and the benefit on the reader. If writers get no credit and only readers are helped, the rational individual doesn’t write. You need something like writing time protected as real work, plus visibility into what got referenced.

“We don’t have enough documentation, let’s have AI write it” can make things worse. Per §4.3, it pollutes search results and raises the cost of finding anything.

Short-term metrics will look worse. Make items smaller and PR count goes up. Raise the bar and lead time initially stretches. Protect learning time and that quarter’s utilization drops. The payoff arrives months later, and in the meantime it looks like things got slower. Explaining and defending that period is a manager’s job, not something team effort can solve.

A subtler problem is that AI output being decent quietly lowers the quality bar. It’s another face of the “almost right but not quite” problem from §1. Work that would have been sent back before now passes, on the grounds that AI produced it. Part of the practical value of writing the bar down is stopping that drift.

One last honest note. This is not a way for a manager to reduce their own workload. For the first few months at least, it increases. Write the bar, get agreement, hold the line, protect the time, explain the dip. The reduction comes after it starts turning.

Summary

The volume AI produces is going to keep rising. That can’t be stopped, and shouldn’t be.

The problem was that pushing that volume straight downstream makes checking heavier faster than the volume grows. Queues climb steeply with utilization (§2.1), detection rates drop as items get larger (§2.2), and unconscious sampling sets in when time runs out (§2.3).

Respond by making checking faster and you’re choosing between lowering unit cost and adding capacity. Both are legitimate, and some measures, like automated gates that eliminate rework, reach the count as well. But as long as upstream generation keeps accelerating, that alone gets overtaken eventually.

What remains is reducing the number of things that must be checked. Two moves. Put small, intent stated, self-verified into the definition of quality, and agree on it in advance. Then make sure work with human judgment in it stays findable by both people and AI.

And don’t spend the time AI freed up on more work. Skip that and utilization rises, the queue lengthens, and nothing else happens. Route the freed time into learning and into maintaining the systems, and the effect applies to everything produced afterward. Finishing one item faster ends with that item.

As DORA keeps saying, AI behaves like an amplifier. Strong organizations get stronger and weak spots break sooner. Fixing the substance before amplification is probably cheaper than checking the amplified output one piece at a time.

You may also be interested in these:

References

References are listed in order, corresponding to the citation numbers in the text.

Other references (not cited by number in the text)

  1. Accelerate State of DevOps Report 2024 - Google Cloud / DORA (2024). Estimated effects of a 25% increase in AI adoption on delivery stability, throughput, documentation quality, and individual productivity. 【Reliability: High】 ↩︎ ↩︎2 ↩︎3 ↩︎4

  2. Announcing the 2025 DORA Report - Google Cloud (2025). AI as an amplifier, the relationship between throughput and instability, and the explanation that acceleration exposes downstream weaknesses. A vendor blog presenting the vendor’s own research. 【Reliability: Medium-High】 ↩︎

  3. 3100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse - Shyam Agarwal, Courtney Miller, Christian Kästner, Bogdan Vasilescu / arXiv preprint (not peer reviewed, 2026). A stratified sample of 3,100 documents drawn from 38,709, used to build a causal model of 26 constructs and 67 relationships. Covers review frequency, merge speed, and discussion volume on AI-authored PRs, along with the instability of those findings. Its central claim is that review is the control point determining the sign of the effect. 【Reliability: Medium】 ↩︎ ↩︎2

  4. Introducing DORA’s inaugural AI Capabilities Model - Google Cloud / DORA (2025). Seven capabilities (a clear AI stance, healthy data ecosystems, AI-accessible internal data, strong version control, working in small batches, user-centric focus, quality internal platforms) and their amplifying effects. A vendor blog presenting the vendor’s own research. 【Reliability: Medium-High】 ↩︎ ↩︎2 ↩︎3

  5. AI-Generated “Workslop” Is Destroying Productivity - Kate Niederhoffer, Gabriella Rosen Kellerman, Angela Lee, Alex Liebscher, Kristina Rapuano, Jeffrey T. Hancock / Harvard Business Review (2025). A survey of 1,150 US full-time desk workers by BetterUp Labs and Stanford Social Media Lab. 【Reliability: Medium-High】 ↩︎

  6. Stack Overflow Developer Survey 2025: AI - Stack Overflow (2025). Over 49,000 respondents across 177 countries. Trust in the accuracy of AI output, frustration with “almost right but not quite,” and time spent debugging. 【Reliability: Medium-High】 ↩︎

  7. Managing Queues in Product Development - An explanation in the product development flow context based on the M/M/1 queueing model. Queue length roughly doubles going from 60% to 80% utilization and climbs far more steeply approaching 95%. The same argument is developed in Donald Reinertsen, The Principles of Product Development Flow (2009). 【Reliability: Medium】 ↩︎

  8. Code Review at Cisco Systems - SmartBear (2006). Analysis of 2,500 reviews and 3.2 million lines over ten months. Thresholds of 200 to 400 lines per review, 500 lines per hour, and 60 continuous minutes. Practical guidance is collected in Best Practices for Code Review. 【Reliability: Medium-High】 ↩︎

  9. Modern Code Review: A Case Study at Google - Caitlin Sadowski, Emma Söderberg, Luke Church, Michal Sipko, Alberto Bacchelli / ICSE-SEIP (2018). Analysis of review logs across 9 million changes, plus 12 interviews and 44 survey responses. 【Reliability: High】 ↩︎

  10. The social economy: Unlocking value and productivity through social technologies - McKinsey Global Institute (2012). Estimates of how interaction workers allocate time, and the reduction available from searchable records. 【Reliability: Medium】 ↩︎

  11. Studying Quality Improvements Recommended via Manual and Automated Code Review - Crupi, Tufano, Bavota / ICPC (2026). Analysis of 240 PRs and 739 comments. AI (GPT-4) caught only about one-tenth of the quality problems humans find, while producing 2.4 times as many comments, roughly 40% of the extra ones being meaningful problems. A complement to human review rather than a replacement. 【Reliability: Medium-High】 ↩︎

  12. A Review of Research on AI-Assisted Code Generation and AI-Driven Code Review - Academic Journal of Science and Technology (2025). A review concluding that LLM-assisted review handles coding conventions and low-severity defects but is weak at detecting high-severity vulnerabilities such as SQL injection and XSS. A review article in a small journal rather than an individual experimental result. 【Reliability: Medium】 ↩︎

This post is licensed under CC BY 4.0 by the author.