Problem Framing & Decision Design13 min read·By , Independent AI Advisor & Fractional AI Lead·

The safest answer your AI team can give you may be “we should not automate this”

The responsible AI decision is not always a deployment. It is choosing the least autonomous design that improves the outcome without transferring hidden costs or authority.

An AI team earns its credibility not only by showing what a system can do, but by identifying the work that should remain unautomated.

Picture a customer-service team trying to shorten the time it takes to resolve a complaint. A new agent can read the customer's history, detect a late delivery, draft an apology, offer a credit and close the ticket. The demonstration looks convincing. The handling time falls. Everyone can imagine the slide in the quarterly review.

Now change one detail. The customer is not asking where the package went. She has spent two weeks being transferred between departments, has already been promised a refund twice, and wants someone to acknowledge that the company has broken its word. The agent can still produce a fluent response. Whether it should close the case is a different question.

This is the decision that gets lost when an organization treats automation as the natural destination of every successful AI experiment. A task can be technically automatable and still be a poor candidate for autonomous execution. The distinction is not an argument against AI. It is an argument for being specific about the result a company is trying to buy.

A serious AI team should be able to return from an with five possible recommendations: stop doing the work, simplify it without AI, assist a person, automate a bounded step, or automate the full workflow. If its recommendations always end in the fifth option, it is not evaluating the problem. It is defending a solution.

What the demo leaves outside the frame

The easiest metric to improve is often the one nearest the model. A faster draft, shorter conversation or larger number of completed actions can be measured immediately. The cost of a broken customer relationship, an unnecessary , a missed learning opportunity or an incorrect decision may arrive elsewhere, weeks later.

A 2026 randomized field experiment in Alibaba's Taobao customer-service operations offers a useful warning. Workers supervising an agent handled AI-eligible chats while continuing to handle other conversations. Average chat duration fell, but ratings for AI-eligible chats were substantially worse. Human intervention helped with some technical ; it was less effective when customers had become frustrated, particularly when escalation happened late. The authors also observed positive spillovers to chats that were not eligible for AI. These are findings from one platform and workflow, not a universal verdict on automated service. They show why speed and customer experience must be measured together. Wang and colleagues, May–June 2026.

This suggests an uncomfortable possibility. A support agent may be efficient precisely because it moves the difficult part of the experience outside the unit of measurement. The human receives the conversation after trust has already been damaged. A dashboard records a successful handoff; the customer remembers being made to explain the problem again.

The question is therefore not whether the model can recognize a complaint or produce a plausible answer. It is whether the proposed operating arrangement improves the whole experience compared with a credible alternative. Sometimes the better alternative is an agent that prepares a concise case history for a human, with the authority to resolve the matter retained by the person who can actually make the customer whole.

That design uses AI. It simply refuses to confuse assistance with the right to decide.

Automation is a business decision, not a model capability

Suppose a finance team asks for an AI system to approve small supplier invoices. The apparent problem is repetitive review. A model can classify invoices, compare line items with contracts and recommend payment. But what if most delays come from suppliers using inconsistent purchase-order references? An improved intake form, a deterministic matching rule and a clear exception owner might eliminate more delay than an autonomous model.

Or take a recruiting team that wants AI to screen candidates. If the actual bottleneck is that hiring managers cannot agree on essential criteria, automation will not resolve the disagreement. It will turn a disputed judgment into a repeatable filter, possibly making the disagreement harder to see.

In both examples, the tempting question is 'Can AI perform this step?' The economically relevant question is 'What changes in the outcome if this step is automated, and what becomes harder to notice, challenge or reverse?'

Recent research gives reasons to separate production activity from delivered value. A May 2026 NBER working paper, revised in September, examines more than 500,000 software developers using a matched observational design. The authors report large increases in coding activity associated with successive AI tools, but much smaller effects further downstream, including releases. It is not a randomized proof that AI caused every reported difference, and software is not every industry. It is evidence that increased intermediate output should not be booked as equivalent final value. Demirer, Musolff and Yang, 2026.

A September 2026 Asian Development Bank working paper adds a different mechanism. In a randomized illustration experiment, access to a text-to-image tool improved output quality at measured points, yet also reduced the incentive to continue working; some participants stopped earlier and submitted weaker final work. The study concerns a specific creative task, not enterprise automation broadly. Its relevance is the design problem: a faster route to 'good enough' can change human effort and therefore the final outcome. Chen and colleagues, September 2026.

A deployment decision should count the work that disappears, the work that moves, and the work that becomes less visible. That is a larger ledger than , hours or completed tickets.

Two measurement paths show AI task speed increasing while final customer or business value can improve, stay unchanged or deteriorate after downstream rework, escalations and trust costs.
Faster intermediate output is a hypothesis about value, not a measurement of value.

The right to say no is a design capability

Some organizations hear 'do not automate' as a failure of ambition. In reality, declining automation can be a precise architectural decision. It may mean the system can gather evidence but cannot authorize a payment; it may draft a response but cannot close a complaint; it may identify a possible policy violation but cannot impose a sanction.

This distinction matters more as agents gain access to tools. The question is no longer limited to whether a generated answer is accurate. It includes which systems the agent can change, what those changes cost, and whether the organization can interrupt an action before it becomes irreversible.

In September 2026, EY published a survey of 202 senior AI decision-makers at large US public companies. Among respondents, 47% said their organizations had at some point bypassed their process for urgent deployments, despite 98% reporting formal AI governance policies. These are executives' self-reports from a specific sample, not an independently audited incidence rate for all firms. The contrast nevertheless illustrates the gap between possessing a policy and operating a control. EY US, September 15, 2026.

On October 29, OpenAI described a layered monitoring system for advanced agents and reported an incident in which a model reached the internet from an environment intended to be isolated. The account is the provider's own technical disclosure, not a general estimate of deployment failure. Its practical implication is narrow but important: even sophisticated model builders treat , monitoring, escalation and human response as separate engineering requirements. OpenAI, October 29, 2026.

An executive cannot outsource that responsibility to a that says 'be careful.' Nor is a human approval button sufficient if the reviewer receives an opaque recommendation after the crucial moment has passed. A June 2026 preprint tested an AI-assisted economic-research workflow across 280 research runs. In that controlled research , the authors report critical failures in 72% of unconstrained runs versus 16% when deterministic computation and binding human decision gates were introduced. The result belongs to that specific research setup, not to all enterprise agents; it illustrates why meaningful authority and reproducible execution matter more than a ceremonial human approval. Zhu, Wang and Zhang, June 2026.

The productive answer is to design the boundary of authority before granting the capability. I have argued elsewhere that an AI agent should receive authority per task, not permissions per role. A recommendation to retain a human decision can be the correct output of that same reasoning.

Five decisions, in order

Before asking whether an AI workflow should go live, I would ask the team to put five alternatives on the same page. They are not stages of maturity. They are competing designs for the same business outcome.

1. Remove the work. Does this activity still need to exist? If a weekly report is never used to change a decision, automating its preparation makes an unnecessary ritual cheaper. Eliminate the report before building a reporting agent.

2. Fix the process without AI. Is the problem caused by unclear inputs, broken handoffs, inconsistent rules or missing authority? Standardized data, a better interface or an explicit exception policy may solve it with fewer dependencies. This option deserves a real cost and outcome estimate, not a token mention in a slide.

3. Assist a person. Can AI reduce the search, drafting or comparison burden while a named human retains judgment? This is often the right choice when the facts are scattered but the consequences require context, empathy or accountability. The person must have enough time and information to make a genuine decision rather than merely endorse the system's recommendation.

4. Automate a bounded step. Can the system act inside a narrow, testable envelope with clear stop conditions? For example, it might match invoices only when purchase order, supplier identity, amounts and contract terms agree exactly; anything else becomes an exception. The envelope must be enforced in code and permissions, not inferred from model confidence.

5. Automate the workflow. Is end-to-end autonomy actually better than the four alternatives on the full outcome, including monitoring, exception handling, rework, risk, customer experience and recoverability? Can a named owner revoke the system's authority and explain what happened? Only then does full automation become a defensible investment rather than the default ambition.

The order matters. Starting at step five turns the other four into objections. Starting at step one turns them into real alternatives.

A five-row decision map compares eliminating unnecessary work, deterministic process repair, AI assistance, bounded automation and full automation, with authority and measurement increasing across the options.
The least autonomous option that improves the whole outcome is the correct starting design.

A practical test for the next AI proposal

For one proposed automation, define a real outcome before the model is evaluated: a complaint resolved without a repeat contact, an invoice paid correctly without extra recovery work, or a decision made with fewer reversals. Choose the measure that the business owner would still care about if the AI product disappeared tomorrow.

Then compare three live alternatives over the same period: today's process, the best plausible non-AI redesign, and the proposed AI-supported design. If full autonomy is under consideration, test an assisted or bounded version as a fourth arm where feasible. Assignment should be randomized when practical; otherwise document selection effects and avoid claiming causal certainty. Include the difficult cases, not just the transactions already known to be easy.

For each alternative, record six things: end-to-end outcome; human time and attention; rework and escalations; customer or employee impact; financial cost including oversight; and the and reversibility of errors. Record who can stop the process, who owns the exceptions and which changes require explicit approval. Averages are not enough when rare errors are expensive.

Decide the in advance. A team should know what result would make it decline deployment, narrow the scope or return authority to a person. A pilot without kill criteria is not an experiment; it is a demonstration waiting for a favorable interpretation.

A reasonable outcome might be: the AI assistant reduces investigation time, but the autonomous version increases repeat contacts after emotionally charged complaints. The recommendation is not 'the AI failed.' It is 'use the assistant; do not automate closure.' That is a successful decision.

Another outcome might be that a deterministic rule outperforms an on invoice matching while costing less to audit. The recommendation is to ship the rule. The team's contribution is the evidence, not the model count.

A third outcome might be that bounded automation wins on speed, accuracy and reversibility, while full autonomy adds little value and increases the tail risk. The right design is a narrow permission envelope, not an argument about whether the organization is sufficiently bold.

What this argument does not claim

There are workflows in which automation clearly deserves to win. Some are high-volume, low-discretion and easy to reverse. Some use AI to perform a task that no affordable deterministic rule can handle. In others, delays and inconsistent human decisions are themselves a major source of harm. Requiring a person to approve every routine action can introduce fatigue, delay and the appearance of accountability without its substance.

The point is not to protect every human task from change. It is to protect the organization's ability to choose the design that produces the best outcome. An automated system with clear boundaries and evidence of superiority is preferable to an inefficient manual process merely because the latter feels familiar.

Nor does one study about customer service, coding or illustration justify a universal rule. The evidence above comes from different populations and methods: a randomized platform field experiment, an observational developer study, a randomized creative-task experiment, an executive survey, a small oversight study and a provider's incident report. They answer different questions. None provides a single percentage to apply to every automation proposal.

What they collectively challenge is the assumption that technical capability, intermediate speed and business value move together.

The recommendation a good AI team should be proud to make

I would rather work with a team that can show why three proposed automations should not proceed than one that announces ten deployments without measuring the alternatives. The first team may be saving the organization from unnecessary complexity. The second may simply be making it harder to admit that the original problem was misdiagnosed.

There is a commercial discipline here. A good investment process does not buy every asset it can afford. A good product team does not build every feature it can imagine. A good AI team should not automate every activity a model can perform.

Its job is to make better operating decisions with evidence, and to keep the organization able to change its mind. Sometimes that means an autonomous agent. Sometimes it means a deterministic rule, a better interface, an informed employee or removing the work entirely.

“We should not automate this” is not the absence of a solution. When the evidence supports it, it is the solution.

Source and evidence notes

Editorial synthesis and the five-option decision test are Berk Bayri's argument, not findings attributed to any single cited study.

Get new essays as they publish.

One email per essay. No noise between.

Berk Bayri

Independent AI Advisor & Fractional AI Lead

I help leadership teams decide where AI belongs, test it before it scales and build the teams that run it, drawing on 25+ years of building digital products, experiences and capabilities.