AI procurement
Buying AI products and services. A team cannot buy 'reasoning'; it buys a configured system with a route, harness and failure behavior, which is what the evaluation should test.
AI procurement is the process of evaluating and buying AI systems. It is harder than conventional software buying because the thing being purchased changes: models are updated, requests are routed differently, harnesses evolve and behavior shifts with the workload.
A procurement team cannot buy "reasoning" or "a model"; it buys a configured system operating under particular conditions. So the useful questions are specific: which exact system will run (runtime address), how does it behave when it fails, who recovers it, and what does that cost?
A better evaluation
- Make the demo fail and examine the unhappy path.
- Ask for a recovery contract.
- Put recovery distance and supervision load into the business case.
- Re-qualify when the route or harness changes.
Read more in Make the AI vendor demo fail.
Related terms
Unhappy path
What happens when an AI system fails, is uncertain or meets a case it was not built for. A buyer should design and test it, because vendor demos show only the happy path.
Recovery contract
A short written artifact in an AI vendor evaluation that states how failures are detected, contained, reversed and handed back to people — and who is responsible at each step.
Runtime address
The full description of the system that was actually evaluated — model version, serving route, harness, tools, precision and operating conditions — not just the model's name.
Used in these essays
Stop adopting AI
AI adoption is rising faster than enterprise value because companies keep installing new intelligence inside old operating models.
The wrong questions about AI right now
Many of the questions that helped us orient ourselves around generative AI are now too blunt to be useful. The harder work is no longer asking what AI is in the abstract, but specifying where it works, where it fails, what authority it should have, and what the whole system costs.
Make the AI vendor demo fail
A polished AI demo proves that a system can succeed under prepared conditions. A buying decision needs different evidence: what happens when the system is wrong, blocked, uncertain or halfway through an action.
The benchmark needs a runtime address
AI systems are increasingly dynamic at runtime. Enterprise evaluation should qualify the serving route, harness, tools and fallback conditions, not just the model name.