No. 7AI Governance
Governance slows AI down: a common AI governance misconception
RealityBad governance creates queues. Good governance predefines boundaries so low-risk actions can move faster without waiting for ad hoc approval every time.
Human in the loop means the system is safe
A common misconception about AI governance, tested against the evidence.
- The myth
- Putting a human approval step in an AI workflow makes the system safe.
- The reality
- A human checkpoint is only a control if the person has the context, competence, time and authority to detect a problem and stop or reverse the action.
Explanation and evidence
The argument
“Human in the loop” describes where a person sits in a process. It does not tell you whether that person can actually control the process.
A reviewer may see too little evidence, receive the request too late, lack domain expertise, face hundreds of approvals, or have no practical way to undo what the system has done. In those conditions, the human can become a ceremonial checkpoint.
NIST’s AI Risk Management Framework treats human oversight as something that must be defined, assessed and documented according to organizational policy. That wording matters. Oversight is not assumed to work merely because a human is present.
OpenAI’s 2026 work on Auto-review shows the opposite failure mode from another direction. Frequent manual approval can create so much friction that users grant broad permissions, write permissive rules or approve actions without fully understanding them. OpenAI explicitly cites reviewer fatigue as one reason manual approval can weaken security in practice.
A person in the workflow is not the same thing as human control.
Why people believe it
The phrase sounds reassuring. It converts an unfamiliar machine-risk problem into a familiar accountability story: the AI proposes, a person decides.
That can work. But only when the decision is genuinely reviewable.
What the evidence says
Good oversight is designed around the decision, not around the presence of a person. The reviewer needs enough context to judge the action, enough time to do so, a clear threshold for escalation, and real authority to deny or reverse it.
Some low-risk actions can be governed more effectively by machine-enforced boundaries and evaluated automated review than by forcing a human to click “approve” repeatedly. Higher-risk decisions may need expert review, dual control or mandatory abstention.
The better question
Instead of asking “Is there a human in the loop?”, ask:
What can this reviewer see, what are they expected to notice, what can they stop, and what happens if they miss it?
That turns human oversight from a reassuring label into a testable control.
Sources
Related reading
The longer argument
Innovation & Capability9 min read
The hidden metric in AI automation is supervision
As AI moves from assisting work to leading it, hours saved stop telling the whole story. The scarce resource shifts to human supervision: approvals, exceptions, context and judgment.
AI Governance & Evaluation13 min read
Your AI agent can follow the objective and still break the company
AI agents can satisfy a local objective while violating the organization’s broader constraints. The control problem is not only model alignment; it is management design.
AI Agents & Systems8 min read
The agent is not the operating model
Giving AI more autonomy does not remove organizational complexity. It gives that complexity permission to act.
Related misconceptions
No. 8AI Agents
Agents remove work: a common AI agent myth
RealityAgents can remove execution work, but they also create supervision, exception handling, evaluation, recovery, permission and maintenance work. The net matters.
No. 4Models & Evaluation
Better models fix bad AI products: a common AI evaluation misconception
RealityA stronger model can improve model-level performance, but product failures often live in context, retrieval, tools, workflow logic, permissions, handoffs, state and recovery.