Why your AI pilot is stuck (and what to do about it)

The call usually goes like this. "We ran a pilot six months ago. The demo was great. Everyone was excited. And now... nothing. The team doesn't know what to do next, and the sponsor has stopped asking."

That is pilot purgatory. It is where AI initiatives go to suffocate quietly, and it is almost never because the technology stopped working. The model that impressed the room in March still works in September. What is missing sits around the model, not inside it.

Here is the idea I want you to leave with: a stuck pilot is stuck on exactly one thing, and it is never the thing the team is currently debating. Find the one thing. Fix it specifically. Do not re-evaluate the approach.

The three places a pilot gets stuck

Every stuck pilot I have seen is in one of three places.

Stage 1: the demo is great, production is silent. The pilot runs against a curated dataset and produces impressive numbers. Then nothing. Nobody knows how to run it against live data, because nobody owns the data pipeline, and the people who built the pilot do not have access to the systems that would let them productionise it.

Stage 2: production works, nobody trusts it. The thing runs. It answers questions, it drafts reports. And every single output is reviewed by the humans it was supposed to relieve, because nobody has written down what "good enough" means. You are now paying for two workflows instead of one, and the second one is the expensive one.

Stage 3: trust exists, budget does not. The business unit wants to scale. The CFO wants to know why the last pilot cost €80k and produced a Slack bot. This is the most salvageable of the three, because it is a narrative problem, not a technical one.

The stages look similar from the outside. The fix for each is completely different, which is why "let's re-evaluate the approach" never works: it tries to fix all three at once and fixes none.

Diagnose it in ten minutes

Sit down with the sponsor and the person who built the pilot. Ask these questions in order. Stop at the first "no".

1. Can the system run against live data today, without anyone from the pilot team touching it? If no, you are in Stage 1.
2. Is there a written definition of an acceptable output, with a number attached, that the business has agreed to? If no, you are in Stage 2.
3. Has that definition been met for at least a month, and are people still reviewing every output by hand? If yes, you are still in Stage 2, and the reviewing is habit, not necessity.
4. Is there a one-page case for scaling, written for the person who signs the cheque, in their vocabulary? If no, you are in Stage 3.

If you got four yeses, the pilot is not stuck. Somebody is simply not making a decision, which is a different essay.

Write the stage down. Say it out loud in the next steering meeting. Naming it is half the fix, because it ends the argument about whether the model is good enough. The model was never the question.

Stage 1: give the pipeline a name

Stage 1 needs an owner for the data pipeline. Not a project manager. Not a consultant. Not "the platform team". A named engineer from the data side whose performance review includes the words "shipped this to production".

Three things to insist on:

• Access before architecture. The owner needs credentials to the source systems in week one. If access takes a quarter, you have learned something important about whether this organisation can run AI in production at all.
• One path, not the perfect path. Batch overnight is fine. A CSV export is fine. Production means "runs on real data without the pilot team", not "elegant".
• A date the owner chose. Not one handed down. People hit dates they set themselves.

If you cannot get a named owner, be honest: the pilot is a vanity project. Kill it, write down what you learned, and spend the money on the next one. A dead pilot with a clear post-mortem is worth more than a zombie.

Stage 2: write the quality contract

Stage 2 needs a quality contract. Literally on paper, one page, signed by the business. It answers the question "when do we stop checking?"

Here is the template I use. Fill in every line; blanks are where the arguments live.

QUALITY CONTRACT: <system name>            Owner: <name>     Date: <date>

What this system does (one sentence):
<e.g. Drafts first replies to inbound supplier queries.>

The five failure modes we care about, and the acceptable rate for each:
1. <e.g. Invents a delivery date>            acceptable: < 1 in 200
2. <e.g. Wrong tone with a key account>      acceptable: < 1 in 100
3. <e.g. Misses a complaint>                 acceptable: < 1 in 500
4. <...>
5. <...>

How we measure them: <sample size per week, who samples, where results live>
What happens when a rate is breached: <automatic action, who is paged>
When manual review stops: <e.g. after four consecutive weeks inside all five rates>
Signed: <business owner>   <engineering owner>

Then build evals that measure those five things and nothing else. When the evals pass for the agreed period, manual review stops. Not "reduces". Stops. Keep a small weekly sample so the numbers stay honest, and retire the daily double-checking.

This is the single highest-leverage intervention in AI adoption, and it is a document, not a model upgrade.

Stage 3: a business case the CFO can read

Stage 3 needs a business case that matches its audience. Finance leaders do not want to hear about prompt engineering. They want to know what this replaces, what it accelerates, and what the unit economics look like at scale.

I have rewritten a dozen of these and they all reduce to the same one-pager:

CONTEXT (one paragraph)
What the pilot does, for whom, and what it has done for the last N weeks in production.

THREE NUMBERS
1. Cost per unit of work today vs. with the system (e.g. per query, per case, per report).
2. Volume at which the system pays back its build and run cost.
3. Quality, in the terms of the quality contract (e.g. "inside all five agreed failure rates for eight weeks").

ONE DECISION
Fund <scope> for <period> at <cost>, owned by <name>, reviewed on <date>.

If you cannot fill in the three numbers, you are not in Stage 3 after all. You are in Stage 2, and the missing quality contract is why the numbers do not exist.

The ownership one-pager

Most pilots die between stages because ownership was never written down. Before the next steering meeting, fill this in and pin it to the top of the project channel:

OWNERSHIP: <system name>

Business owner (decides what "good" means):        <name>
Engineering owner (keeps it running):              <name>
Data owner (source systems and access):            <name>
Who approves changes to prompts or rules:          <name>
Who gets paged when a failure rate is breached:    <name>
Next review date, and what would make us stop:     <date> / <condition>

Every line must be a person, not a team. If a line says "TBD" for more than a week, that line is your Stage.

The ninety-day rule

If a pilot has been stuck for more than ninety days, the problem is never the model. It is ownership, quality definition, or narrative. Pick the one that is broken and fix it specifically.

Do not "re-evaluate the approach". Do not "explore alternatives". Do not commission a second pilot to compare. Specific fixes for specific problems. Everything else is theatre, and expensive theatre at that.

What to do on Monday

1. Run the ten-minute diagnosis with the sponsor and the builder. Write the stage on the whiteboard.
2. Fill in the ownership one-pager. Replace every team name with a person's name before you leave the room.
3. If Stage 1: get the data owner credentials this week, and a date they chose.
4. If Stage 2: draft the quality contract with the business owner. Five failure modes, five numbers, one stop condition. Sign it.
5. If Stage 3: write the one-pager in the CFO's vocabulary. One paragraph, three numbers, one decision. Book fifteen minutes to walk through it.
6. Put the next review date in the diary, with the condition that would make you stop.

A stuck pilot is not a technical failure. It is an unmade decision wearing a technical costume. Take the costume off and the decision is usually obvious.