Why most AI pilots never ship — and what the survivors do differently

By 2026 the numbers are common knowledge among buyers: the large majority of enterprise AI pilots never reach production, and Gartner expects over forty per cent of agentic AI projects to be cancelled by the end of 2027. The interesting part is why — because the cause is almost never the thing teams spend their time arguing about.

When the failures are analysed, they cluster around operating-model problems, not model quality: unclear success criteria, insufficient access to the data and tools the agent actually needs, and evaluation drift — the system quietly getting worse with no one watching the right number. None of those are solved by a better model. They are solved by deciding, up front, what "working" means and how you will know.

This is good news, oddly. A model-quality ceiling would be out of your control. An operating-model failure is entirely within it.

The pilots that survive are not the ones with the cleverest demos; they are the ones built like production software from the first week.

• They define success criteria before building. A specific, measurable target — handling time, error rate, deflection — agreed with the business, not discovered afterwards.
• They build the evals first. Golden datasets and scorers that run in CI, so a regression is caught by a gate, not by a customer.
• They solve data and tool access early. Most agents fail because they cannot reach the system that holds the answer — and giving them raw credentials is not the fix. A gateway is.
• They keep humans on the decisions that matter. The agent moves the work; ownership of the consequential call stays legible and human.
• They stay model-agnostic. The model is a swappable part. Coupling a business process to one vendor's model is a risk, not a strategy.

When you get a demo and something works 90% of the time, that's just the first nine.

— Andrej Karpathy, on the gap between demo and product

We only take AI work we believe can reach production. That means the unglamorous parts — success criteria, evaluations, governance, observability, a twelve-month view of what good looks like — are the start of the engagement, not an afterthought. If a problem cannot be framed that way, the honest answer is that it is not ready, and we will say so. In 2026, that is a more credible position than enthusiasm.