The argument
A bridge engineer can tell you the load a bridge was built to carry. A drug maker has to list known side effects before the product reaches a pharmacy.
It is natural to assume the companies behind large AI models hold the same kind of knowledge about what they ship. They do not, and several of them say so openly.
These models are not written instruction by instruction. They are trained on enormous amounts of data, and what they end up able to do is an outcome of that training, not a specification someone typed in. Builders control the ingredients and the process. They do not get a complete map of the result.
So the knowledge comes afterwards. Teams test a finished model to find out what it does well, where it fails and what it should not be allowed to do. That testing is serious work. It is also, by its nature, a sample.
The builders know how the model was made. What it can do is something they are still finding out, just earlier than you.
Why people believe it
Most technology we buy has been designed to a specification, and vendors speak about AI products in the same confident language: the model "can" summarise contracts, "will" catch fraud, "does not" leak data.
Release notes, model cards and benchmark tables add to the impression of a fully characterised product. They are useful documents. They describe what was measured, which is not the same as everything that is there.
What the evidence says
Anthropic's chief executive, Dario Amodei, wrote in April 2025 that people outside the field are often alarmed to learn that "we do not understand how our own AI creations work." In the same essay he described models showing unexpected emergent behaviours that builders struggle to predict or rule out.
Reporting from Axios in September 2026 described the leading labs as still racing to understand their newest systems, with OpenAI's chief executive acknowledging that no one fully understands the consequences of how capable the models are becoming.
Research on capability testing points the same way. Evaluations are meant to estimate what a model can do, yet abilities that were present but not visible have repeatedly been drawn out of models, sometimes long after release.
None of this means the models are uncontrolled or that vendors are hiding something. It means the honest description of a model is "this is what we found", not "this is all it does".
The better question
Do not ask a vendor only what the model can do. Ask how they found out, what they have not tested, and what they learned about it after release.
Then test it yourself on your own data and your hardest cases. Where a decision cannot be undone, about someone's health, money or job, keep the final decision with a person who can explain it.