Berk Bayri

Inference

Running a trained AI model to produce an output for a request, as opposed to training it. Inference cost is the model-side spend that is easy to see but only one part of total AI cost.

Inference is the stage where a trained model is used: you send a request and it computes a response. It is distinct from training, where the model's weights are learned. Every time a user asks a question, an agent calls a tool or a classifier scores an input, inference happens.

A production request can pass through much more than one model call. It may go through a router, retrieve context, call tools, use a particular inference configuration, invoke another model, hit a permission boundary, retry, fall back or be escalated to a person. The route and configuration therefore matter as much as the model name: see serving route.

Inference cost

Inference cost is the model-side spend, usually driven by tokens. It is the most visible line in an AI budget and the easiest to cut, which is the risk: lowering it can raise review, exception and recovery costs elsewhere. That is what the cost-migration test checks, as part of total cost of ownership.

Read more in A cheaper AI model can move the cost instead of removing it.

Used in these essays

Problem Framing & Decision Design5 min read

The most valuable AI project may be a policy change, not a product

Before you fund an AI system to handle a slow process, find the rule that created the work. Changing the rule can beat automating it.

AI Strategy & Transformation7 min read

The question your board should ask about AI is not “what are we doing?”

A list of AI projects tells a board what management chose to launch. It does not show where AI already shapes decisions nobody approved.

Innovation & Capability11 min read

Stop asking what AI can do. Ask what your company can absorb.

The gap that matters is between what AI can produce and the changes an organization can reliably absorb into ordinary work.

AI Strategy & Transformation12 min read

Your AI agent estate is becoming a capital allocation problem

Once agents multiply across workflows, the management problem changes: which agents deserve more capacity, which should be merged or constrained, and which should be retired?

AI Agents & Systems14 min read

OpenAI Dots is a test of whether AI can carry a goal, not just complete a task

Dots matters less as another capable assistant than as a test of persistent delegation: can AI keep carrying a goal without giving the user a new system to manage?

Problem Framing & Decision Design10 min read

Cahit Arf’s 1959 question about thinking machines

In a 1959 text on thinking machines, Cahit Arf moved from clocks and relays to a harder question: what happens when a machine meets a problem that was not anticipated when it was built?

Problem Framing & Decision Design14 min read

The wrong questions about AI right now

Many of the questions that helped us orient ourselves around generative AI are now too blunt to be useful. The harder work is no longer asking what AI is in the abstract, but specifying where it works, where it fails, what authority it should have, and what the whole system costs.

AI Strategy & Transformation9 min read

A cheaper AI model can move the cost instead of removing it

A lower model bill can hide a higher workflow bill. The useful AI TCO question is not only what got cheaper, but where the cost moved.