Counterfactual
An estimate of what would have happened without an intervention, used to distinguish its effects from changes that would have happened anyway.
A counterfactual is what would reasonably have happened without the intervention being evaluated. It provides a comparison for separating a genuine effect from changes that would have happened anyway.
If a support team handles requests faster after an AI rollout, that does not establish that AI caused the improvement. Staffing, policy, seasonality or case mix may have changed. Randomized controls, staged rollouts, matched groups or documented baselines can provide comparisons with different levels of confidence.
The counterfactual is an estimate, not an observable parallel universe. It should make its assumptions and uncertainty visible.
Related: ROI, productivity is upstream of value, and cost per accepted outcome.
Related terms
ROI
Return on investment: the value gained relative to the cost. For AI it is easy to overstate by counting local savings and ignoring supervision, exceptions and recovery.
Productivity is upstream of value
Individual time savings from AI do not automatically become business value; they do only if the workflow, decisions and economics change downstream to use the freed capacity.
Cost per accepted outcome
The total cost of getting one AI-assisted result that is actually accepted and used, counting model spend, retries, human review, exceptions and recovery — a truer measure than cost per call.