A common misconception about AI models and evaluation, tested against the evidence.
A system that always answers can look more capable than a system that sometimes stops. That comparison is often backwards.
OpenAI's research on hallucinations makes the incentive problem unusually concrete. When models are scored mainly on whether they produce the correct answer, guessing can outperform abstaining even when the guess is wrong most of the time. In one SimpleQA comparison cited by OpenAI, an older model had slightly higher raw accuracy but a dramatically higher error rate because it almost never abstained.
That is not merely a benchmark curiosity. In a business process, the cost of a confident wrong answer can be much larger than the cost of asking for clarification, retrieving more evidence or escalating to a person.
A system that answers less can make better decisions if it knows which answers are too expensive to guess.
Chat interfaces teach us to treat silence or refusal as failure. Users ask; software responds.
But decision systems have another option: do not decide yet.
OpenAI and Anthropic evaluations show a real tradeoff between coverage and confident error. Some models answer more questions; others refuse more often and hallucinate less. Neither extreme is automatically best. The right operating point depends on the consequence of being wrong.
For a low-stakes brainstorm, guessing may be acceptable. For a payment, medical recommendation, compliance decision or irreversible agent action, uncertainty should change what the system is allowed to do.
Do not ask only, "How often does the AI answer correctly?"
Ask: When should it answer, when should it retrieve, when should it ask, and when should it abstain?
That is not a limitation to hide. It is part of the product.
AI systems are increasingly dynamic at runtime. Enterprise evaluation should qualify the serving route, harness, tools and fallback conditions, not just the model name.
A polished AI demo proves that a system can succeed under prepared conditions. A buying decision needs different evidence: what happens when the system is wrong, blocked, uncertain or halfway through an action.
OpenAI's new misalignment disclosure framework exposes a useful enterprise design principle: record anomalous AI behavior before the organization has finished explaining it. Otherwise incident systems quietly become filters for what teams already understand.
RealityA human checkpoint is only a control if the person has the context, competence, time and authority to detect a problem and stop or reverse the action.
RealityA stronger model can improve model-level performance, but product failures often live in context, retrieval, tools, workflow logic, permissions, handoffs, state and recovery.
RealityUseful context is selective. More tokens can bury relevant evidence, preserve stale assumptions and increase the amount of contradictory material the agent has to resolve.