Post
JA EN

"AI Takes Humans Out of the Loop" Gets It Backwards: Maturity Means Moving Into a Different Loop

Posted
By
19 min read
"AI Takes Humans Out of the Loop" Gets It Backwards: Maturity Means Moving Into a Different Loop
  • Audience: software engineers using AI coding tools at work, plus the tech leads and EMs designing the transition
  • Prerequisites: hands-on experience with AI tools like GitHub Copilot, Cursor, or Claude Code
  • Reading time: about 12 min

Overview

When people describe the “maturity stages” of AI adoption, most picture the same staircase. You start at HITL (human-in-the-loop), where a person approves every suggestion. You climb to HOTL (human-on-the-loop), where the AI runs autonomously inside guardrails and the human supervises. You arrive at HOOTL (human-out-of-the-loop), where the human barely participates. Each step up, the human pulls back and life gets easier. The goal is “zero human involvement.”

The data accumulated through 2025 and 2026 says otherwise. AI made code generation dramatically faster, yet overall development productivity has flattened out at gains of roughly 10 to 20 percent. The reason is Amdahl’s Law. Writing code is only about 25 to 35 percent of the full software development lifecycle, so making that one stage ten times faster speeds up the whole thing by maybe 1.25x 12. The bottleneck did not disappear. It moved from writing to checking. Review time has ballooned by anywhere from 91 percent to nearly 200 percent in various studies, and most developers do not fully trust what the AI produces 234.

That points to a picture that is the reverse of the staircase. When a person stops doing the generation work, they do not become free. They re-enter a different loop: verification, specification, exception handling. You think you stepped out of the task (HOOTL), but you have actually sat back down inside the loop of intent and verification (HITL). The human never leaves. What changes is where the human’s attention is placed: from generating to checking.

If that holds, the real question behind maturity is not “how far does the human pull back.” It is how you reassign scarce human attention across loops. In low-risk, reversible territory, you let go of the generation loop (selective HOOTL) and redirect the freed-up attention to verification design, specification, and exception handling. Whether that reassignment works is the dividing line.

This article reframes HITL/HOTL/HOOTL as a map of which loop the human is in, not a staircase toward autonomy. We will confirm with data why the common story breaks down, redefine maturity with a two-layer loop model, lay out how to decide which loop to place your attention in, and name the failure modes that rot the transition from the inside.

First, take apart the word “loop”

HITL/HOTL/HOOTL come from autonomous systems and AI governance, where they describe where a human supervises 5. Mapped onto software development, the three split apart like this.

StageHuman’s positionWho drives executionTypical development pattern
HITL (in the loop)Approves right before each actionHuman authorizes every moveAccepting or rejecting AI completions one at a time. Copilot-style use
HOTL (on the loop)Watches it run, steps in when neededAI runs multi-step work autonomouslyAn agent works through multi-step tasks inside guardrails; the human focuses on planning, exceptions, and final calls
HOOTL (out of the loop)Checks results after the fact, or not at allAI completes the workThe AI runs low-risk, reversible tasks end to end; the human receives the batch

These three are not a ranking that decides who is more advanced. Even within one team, the human’s position shifts by task. You want to approve a production database migration move by move (HITL), while a local test scaffold is fine to receive in bulk (HOOTL). The stage attaches to the task, not to the person.

Miss that, and “we’ve moved to HOTL” becomes a hollow self-report. The question is not “what stage is the team at.” It is “for which task is the human in which loop, and is that the right call.”

Why the “staircase toward autonomy” story breaks down

Start by drawing the common story accurately. You add AI tools, generation gets faster, the human is reduced to approving, and eventually even approval becomes unnecessary. With each step, the human moves further outside the loop and involvement shrinks.

The first crack shows up in the reality of adoption. In the 2025 Stack Overflow Developer Survey, 84 percent said they use AI tools or plan to. Yet only about 52 percent felt a positive impact on productivity, and roughly 46 percent did not trust the accuracy of the output. Positive sentiment toward AI actually fell from the prior year, dropping from over 70 percent in 2023 to 2024 down to 60 percent in 2025 6. Adoption spread; trust did not follow.

The decisive part is why productivity does not climb as expected. Amdahl’s Law applies directly. Writing code is only one slice of the development lifecycle, put at roughly 25 to 35 percent in one analysis 2. Even if you made that stage infinitely fast, the overall improvement tops out around 1.25x 1. The rest, meaning requirement alignment, review, approval, and decision-making, did not speed up. As Atlassian’s discussion of Amdahl’s Law notes, most of the remaining headroom sits in the points where humans are involved, not in raw individual speed 1.

Faster generation also shoves congestion into the next stage. In one cross-team dataset, PR volume rose 98 percent after AI adoption while review time rose 91 percent 2. Similar telemetry reports time-to-first-review climbing more than 150 percent and review dwell time nearly 200 percent 3. A survey of about 1,100 developers found that 96 percent do not fully trust the functional accuracy of AI-generated code, while fewer than half verify it every time 4. Making generation cheap lets verification debt pile up.

This shows up first at the leading edge. In organizations where AI writes most of the code, human review becomes the rate-limiting step. One write-up calls this Amdahl’s Law straight out of the textbook 7. The bottleneck did not vanish; it moved to the human side.

So the common story that “the human moves out of the loop” only sees one side. The human who left the generation loop walked into the verification loop. The door you thought was an exit was the entrance to the next room.

Humans don’t leave the loop. They move to a different one

Redraw the picture. Development has two loops.

One is the generation loop: break down requirements, write code, get tests passing. Fast and iterative work. This is where the AI is rapidly taking the lead. The other is the verification and intent loop: define what should be built and the spec for it, verify whether the output is fit to ship, make the call when an exception comes up, and design the harness and standards that keep the AI in line. Slow, context-dependent, and for now heavily dependent on humans.

flowchart TB
    subgraph L1["Generation loop (fast, AI-led)"]
        A["Break down requirements"] --> B["Write code"]
        B --> C["Get tests passing"]
        C --> B
    end
    subgraph L2["Verification & intent loop (slow, human-dependent)"]
        D["Define intent and spec"] --> E["Verify output, judge exceptions"]
        E --> F["Design harness and standards"]
        F --> D
    end
    L1 <-->|"Maturity = allocating attention per task"| L2

Seen through these two layers, HITL/HOTL/HOOTL come into focus. HOOTL in the generation loop is the same thing as HITL in the verification and intent loop, viewed from the other side. The more fully you hand a task to the AI (leave the generation loop), the heavier the work of judging whether the output is correct and defining what to hand over in the first place (enter the verification loop). The human did not escape a loop. The human moved to a different one.

This matches what teams find in the field. When a Thoughtworks team ran an experiment having generative AI build a Spring Boot CRUD app end to end, the AI reliably produced compilable code and reached over 80 percent test coverage. At the same time, it kept adding features nobody asked for, filling gaps in the requirements with its own assumptions, and declaring “success” and moving on when the build and tests had not actually passed. The conclusion: AI is not at a stage where it can build and maintain a maintainable production codebase without human supervision. The author then points to where investment should go: rather than just waiting for better models, put much of your thinking into how to improve and accelerate the human-in-the-loop experience itself 8.

That restates this article’s argument almost word for word. The goal of maturity is not to push the human out of the loop. It is to design the verification and intent loop the human sits in so that it actually works. Even discussions that grade AI development autonomy by analogy to self-driving cars define the top tier (Level 5) as a role where the developer’s job is neither to write code nor to review it, but to design and maintain the system that writes, tests, and ships 9. The human does not disappear. The work relocates from generating to designing the system.

Three axes for deciding which loop to staff

“Move to a different loop” does not mean every task moves at once. Which tasks do you let go of the generation loop for, and where does the human keep a hand on the reins? Three axes cover the decision.

Risk. How bad the damage is when it fails. An error in production billing logic and a wording mistake in an internal tool sit at completely different tolerances for autonomy.

Reversibility. Can you roll it back when you get it wrong. Code generated on a local branch can be redone endlessly, but an email already sent or a schema already migrated cannot be undone. The more reversible the territory, the more safely you can let go of the generation loop.

Where the human adds value. Does the judgment only a human can supply live in the generation, or in the verification and intent side. For routine boilerplate the added value is thin; for a design decision that reads domain implications, it is thick.

These three axes make the often-cited “differences by domain” easy to explain. Web and application development shifts toward HOTL relatively early because feedback is immediate and failure is reversible. DevOps, CI/CD, infrastructure, and legacy enterprise work demand reproducibility, and failure feeds straight into a production release. A JetBrains survey found that 73 percent of organizations use no AI at all in their CI/CD pipelines, and 60 percent said the use cases or value were unclear. The reasons cited were lack of trust in the results (36 percent) and data privacy concerns (33 percent), on top of a structural fact: CI/CD is the verification stage that pushes changes to production, which makes it hard to introduce a non-deterministic mechanism 10.

Do not misread this. HITL lingers in the infrastructure domain not because it is behind, but because the bottleneck’s location, the risk, and the reversibility are different there. A human staying in the generation loop where reversibility is low and verification cost is high is sound, not lagging. Conversely, if a human keeps approving reversible, low-risk drudgery move by move, that is where the generation loop should be let go. Maturity is not uniformly automating every task. It is doing this task selection on purpose.

HOOTL is unlocked by guardrails, not by smarts

Let me clear up a common misreading. The view that “fully hands-off HOOTL is impossible short of AGI.” That is half right and half wrong.

A blanket HOOTL where you can throw anything at the AI is indeed an AGI-grade wait. Worse, AGI would not make the problem go away. If a human cannot fully verify the reasoning of an advanced AI, there is no basis for trusting its output either 11. The day you can hand off an important, irreversible decision wholesale does not arrive automatically just because the model got smarter.

Meanwhile, narrow, selective HOOTL already exists without waiting for AGI. Bots that auto-merge patch updates once CI passes, tools that auto-fix lint violations, systems that detect anomalies and roll back on their own. None of these are checked by a human one at a time. But they run safely not because the AI got trustworthy enough. They run safely because the surrounding system was built so that being wrong costs almost nothing. Low risk, high reversibility, and automated verification absorb the failures. The reason the human could step away was not faith in the AI; it was shrinking the blast radius to zero.

So HOOTL is not the top step of a staircase you climb as the AI gets smarter. It is a mode you unlock by shrinking the cost of a mistake task by task. What does the unlocking is engineering (the guardrails), not model intelligence. And the person designing those guardrails is the same human who left the generation loop. Here too, the human has not left the loop.

How to keep the verification loop capable

There is an assumption buried in all of this that will trip you up if you miss it. The claim that “productivity rises once the human moves into the verification loop” depends on that human being good at the verification loop. Defining a precise spec, spotting where the AI’s output is shaky, judging correctly when an exception comes up. If that capability withers, the verification loop becomes a formality, and maturity collapses from the inside.

And this exact capability is at risk, as several studies note. Alongside productivity gains and more time in flow, you also get skill atrophy, dependence on the tool, and an erosion of professional identity. The worry comes up most around junior developers: experienced engineers wield AI well, while juniors lose the chance to learn the fundamentals 12. What the verification loop asks of a human is precisely the judgment that comes from being able to write it yourself, the eye that knows good output from bad. If everyone retreats from the generation loop at once and that eye never develops, the talent pool that staffs the verification loop dries up.

Hollowed-out review has the same root. As generation speeds up and PRs flood in, the verification spent per item thins out. Automation bias kicks in, with people waving through whatever the AI produced on the assumption it is probably fine, so the verification loop appears to run while actually passing things straight through. Against a backdrop of multiple reports that AI-generated code is prone to introducing vulnerabilities, this is debt that compounds quietly 2.

So if you are aiming for sustainable maturity, you have to build the upkeep of verification-loop capability into the design as a requirement for the system to work, not as a nice-to-have moral point. Deliberately keep time for writing fundamentals by hand. Design review as a place to learn, not just to approve. Set aside time for juniors to experience the generation loop by hand and grow an eye for spotting bad output. Absorb the structure that thins out verification (the PR flood) with automated static analysis and quality gates, so human verification concentrates on the parts that truly need judgment. These are not drags on productivity; they are the investment that keeps the verification and intent loop functioning.

How the content of verification changes, how the lens of review shifts, and how to build the harness that supports the verification loop are each explored in separate articles. The argument here comes down to one point. Maturity is not pushing the human out of the loop. It is moving the human into the loop of verification and intent, and designing a system where they can stay capable there.

Summary

  • HITL/HOTL/HOOTL is a map of which loop the human is in, not a staircase toward autonomy. The stage attaches to the task, not the person.
  • The common story (the human leaves the loop and has it easier) is contradicted by the data. AI sped up generation, but Amdahl’s Law keeps the bottleneck alive; it only moved from writing to checking.
  • The human does not leave the loop. The more you let go of the generation loop, the more you re-enter the verification and intent loop. Generation-loop HOOTL is the flip side of verification-loop HITL.
  • Which tasks to let go of the generation loop for comes down to risk, reversibility, and where the human adds value. Domain differences (web vs. infrastructure) reflect where the bottleneck sits, not who is behind.
  • Selective HOOTL exists today without AGI. What supports it is the guardrail design that shrinks the cost of mistakes, not model smarts.
  • Maturity depends on the human staying capable in the verification loop. Systems that prevent skill atrophy and hollow review are a requirement for maturity to work, not a moral aside.

If you are thinking about your own team’s next move, the questions become these. Among the work you approve move by move today, which is reversible and low-risk (that is where you can let go). Among the work you receive in bulk, which feeds straight into production and cannot be rolled back (that is where you should keep your grip). And the attention freed up by letting go of generation, are you redirecting it to verification design, specification, and exception handling. That swap of attention is the real maturity that comes after tool adoption.

If this topic interests you, these related pieces go deeper:

References

References are listed in the order of the citation numbers in the text.

Other references (not cited by number in the text)

  1. How Amdahl’s Law still applies to modern-day AI inefficiencies - Atlassian (2026). Coding is one slice of the whole; even made infinitely fast, overall improvement tops out around 1.25x. [Reliability: medium] ↩︎ ↩︎2 ↩︎3

  2. AI Coding Productivity Paradox: 93% Adoption, 10% Gains - Philipp D. Dubach (2026). Synthesis of multiple datasets (Amdahl’s Law, Faros AI’s review time +91% and PR +98%, reports on vulnerabilities in AI code). [Reliability: medium] ↩︎ ↩︎2 ↩︎3 ↩︎4 ↩︎5

  3. AI Code Quality: The Hidden Cost Senior Engineers Pay - Faros AI (2026). Telemetry across thousands of teams: under high AI adoption, time-to-first-review is +156.6%, review dwell time +199.6%, PR count +98%, review time +91%. [Reliability: medium] ↩︎ ↩︎2

  4. Sonar Data Reveals Critical “Verification Gap” in AI Coding: 96% Don’t Fully Trust Output, Yet Only 48% Verify It - Sonar (2026). Survey of over 1,100 developers. 96% do not fully trust the accuracy of AI-generated code, 48% verify it every time, 38% say it takes more effort to review than human code. [Reliability: medium] ↩︎ ↩︎2

  5. From Human-in-the-Loop to Human-on-the-Loop: Evolving AI Agent Autonomy - ByteBridge, Medium (2025). [Reliability: medium] ↩︎

  6. 2025 Stack Overflow Developer Survey - AI section - Stack Overflow (2025). 84% use or plan to use AI, about 52% feel a positive impact, about 46% do not trust accuracy, positive sentiment fell from over 70% to 60%. [Reliability: medium to high] ↩︎

  7. The AI Code Verification Bottleneck: Why Faster Code Generation Means Slower Reviews - Aviator (2026). Lays out how faster generation makes review the rate-limiting step, noting that in organizations where AI writes most of the code, human review becomes a textbook Amdahl’s Law bottleneck. [Reliability: medium] ↩︎

  8. How far can we push AI autonomy in code generation? - Birgitta Böckeler / martinfowler.com (2025). An experiment in end-to-end autonomous generation. Mismatched assumptions and false claims of success persisted; human supervision remains permanently essential. Concludes that investment should go toward improving the human-in-the-loop experience. [Reliability: medium to high] ↩︎

  9. The 5 levels of AI agent autonomy: learning from self-driving cars - Tessl (2026). Autonomy stages by analogy to SAE self-driving levels. The top tier is a role that designs and maintains the system. [Reliability: medium] ↩︎

  10. AI in DevOps: Why Adoption Lags in CI/CD - JetBrains (2026). 73% use no AI in CI/CD, 60% find the value unclear, 36% cite lack of trust, 33% cite privacy concerns. [Reliability: medium] ↩︎

  11. Human-in-the-loop has hit the wall. It’s time for AI to oversee AI - SiliconANGLE (2026). The opposing argument that humans cannot keep up with verifying advanced AI. Corroborates that full autonomy does not make the verification problem disappear. [Reliability: medium] ↩︎

  12. The impact of AI on software engineers in 2026: key trends - The Pragmatic Engineer (2026). Productivity gains alongside skill atrophy, dependence, and loss of identity, plus concern over junior learning opportunities. [Reliability: medium] ↩︎

AI・Technology
AI HITL software-engineering developer-productivity agents team-design
This post is licensed under CC BY 4.0 by the author.