Why is the second AI project harder than the first?

Because the first is usually built from nothing and the second is not. A first project typically has a new interface, data prepared for the purpose, no existing users, and no integration requirements, while the second meets legacy data, an existing permission model, third-party systems, and users who already have expectations about how the product behaves.

Does a successful AI pilot predict a successful rollout?

Only weakly, because the two measure different things. A pilot measures what the technology can do under conditions you chose, while a rollout measures what your organisation can handle, and the second question involves data quality, integration reliability, and user tolerance that the pilot deliberately excluded.

What usually consumes the time in a second AI project?

Data reconciliation and integration work, rather than model work. Existing records are inconsistent, duplicated, and carry fields whose meaning changed over the years, and connecting to systems you do not control introduces failure modes such as silent partial success that need handling before anything is released.

Why do users reject a second project that is technically better?

Because they compare it against the behaviour they already know and read every difference as a defect. A first project has no existing system to be measured against, so the same output is judged as new capability, while in a second project it is judged as a change to something that already worked for them.

How should you scope a second AI project differently?

Scope it on the integrations, the data condition, and the permission model rather than on the model work. Those three areas hold most of the uncertainty in a second project, so estimating from the first project's timeline will produce a plan that is wrong in a predictable direction.

What is the demo-to-production gap?

The distance between a system that works on prepared inputs in a controlled setting and one that keeps working with real inputs, real volumes, real permissions, and real failure conditions. It is the majority of the effort in most AI features, and it is invisible in a demo because a demo is designed to avoid exactly those conditions.

Why does team enthusiasm drop on the second project?

Because the team reads the slowdown as their own failure rather than as a change in the problem. Nobody told them the first project had been running with several constraints removed, so when progress slows they conclude they were lucky before, which is a damaging and inaccurate story.

What should you tell a sponsor before the second project starts?

That the second project is a different kind of problem and will take longer per unit of visible progress, with the reasons named specifically. Explaining what to expect in advance is a plan, while explaining it after the slowdown arrives sounds like an excuse regardless of how true it is.

What integration failures should you expect in a second project?

The quiet ones, especially a system that returns a success response and does not perform the action. Timeouts and outright errors are straightforward to handle because they are obvious, while silent partial success corrupts state and is usually discovered days later by someone reconciling records.

Is it worth doing a pilot built from nothing at all?

Yes, provided the results are read for what they are. A pilot establishes whether the approach can work and gives the team real experience with the tools, and the mistake is treating its timeline as a baseline for work that has to meet existing data, systems, and users.

Why the second AI project is harder than the first: Reveneau

Q: How do permissions change between a first and second project?

In a pilot everyone with access to the feature is usually entitled to all the data in it, so access control is one check at the start. A second project spans data that different people are entitled to see in different amounts, which turns permissions into an architectural decision affecting retrieval, indexing, and logging.

There is a pattern in how organisations adopt AI, and the second step of it surprises almost everyone.

The first project goes well. It is released, it demonstrates something real, the sponsor is pleased, and the team believes it has found a good way of working. The second project is chosen with more confidence and more ambition, and it stops making progress. Timelines are delayed. The demo that arrives three months in is worse than the one from the first project at three weeks. And the conclusion that forms, usually unspoken, is that the team got lucky the first time.

They did not get lucky. The problem changed, and nobody wrote down how.

What the first project was quietly missing

Look at how first projects get selected. Someone picks the idea that can be built without asking permission from four other teams. That selection is sensible and it is also a filter, and the filter removes precisely the constraints that make software hard.

No existing users. Nothing to compare against, no workflow to preserve, nobody whose Tuesday morning changes because of it.

Clean data. Usually a set assembled for the purpose, or a corpus that was already tidy. It was prepared, and preparation is invisible in the result.

No integration. The pilot stands alone. It reads and it writes to its own store, and nothing else in the company depends on what it does.

Simple permissions. Everyone with access to the pilot is entitled to everything inside it, which means access control was one check at the start rather than a design constraint.

No consequences. If it produced a wrong answer, someone laughed and tried again. Nothing was invoiced, dispatched, or deleted.

Every one of those is a constraint that was not present. None of them appears in the retrospective, because retrospectives record what happened and these are things that did not.

So the first project answered a real question: can this technology do the thing? What it did not answer, and could not, is whether the organisation can support it. That second question is the one the second project asks, all at once, on day one.

What arrives in week two of the second project

The data is real now. Records entered by hundreds of people over a decade, with duplicates, inconsistent formats, and fields whose meaning quietly changed in 2019 when a process was redesigned. Nobody documented the change because everyone involved knew about it. Two of those people still work here. This is not a data cleanup task that precedes the project. It is a substantial part of the project, and it will raise a business question nobody has an owner for: which of these two records is the customer.

Permissions became architectural. The second project spans data that different people are entitled to see in different amounts. That decision drives the retrieval design, the indexing strategy, and the logging, and it cannot be added later without rebuilding the retrieval layer. The full explanation is in AI agent permissions and guardrails.

Integrations fail in ways your code cannot. You now depend on systems you do not control. The straightforward failures, timeouts and errors, are obvious and are handled in an afternoon. The expensive one is silent partial success: a call returns a success response and does not perform the action. State diverges, nothing alerts, and somebody discovers it four days later while reconciling records. Planning for that class of failure is most of what makes an integration reliable, and it is in making AI agents reliable.

Users have expectations. This is the underrated one. A first project has no existing system to replace, so its output is judged as new capability. A second project replaces or sits beside something people already use, so every difference reads as a defect, including the improvements. "It used to sort them by date" is a complaint even when the new ordering is better, because the person had a workflow built on the old one.

Why the team loses motivation

The slowdown is not the damaging part. The story people tell about it is.

An engineer who released the first project in six weeks and is twelve weeks into the second one, with less to show, does not conclude that the second problem is harder. They conclude something is wrong with them, or with the team, or that the first result was an accident. That story spreads quietly and it is damaging, and it is false.

The METR study from July 2025 is worth knowing about here. Sixteen experienced open-source developers worked on their own repositories with and without AI assistance. They expected to be 24 percent faster. They were 19 percent slower, and still believed afterwards that they had been 20 percent faster. Sixteen people is a small study and the authors are careful about it. The finding that matters for this discussion is the gap between felt speed and measured speed, in both directions. If perception can be off by 39 percentage points against measured time, then a team's sense that it has become slower and worse is not evidence of anything either.

The 2025 DORA report, from around 5,000 respondents, describes the underlying mechanism: "AI doesn't fix a team; it amplifies what's already there". A first project has almost no organisational problems to make larger. A second project has all of it, so unclear ownership, slow decisions, and undocumented data all become visible at once, and they arrive looking like engineering problems. We wrote about that in AI is an amplifier, not a fix.

Four things that change the outcome

Scope on the integration and the data. Estimate the second project by looking at the systems it touches and the condition of the records it reads, not by scaling the first project's timeline. The model work is usually the small part. The gap between a working demonstration and a system that works with real inputs is covered in the AI last mile, and its shorter form is in an AI demo is not a product.

Write the checks before the build. The first project could be evaluated by a person looking at it, because nothing was at stake. The second one has consequences, which means correctness needs to be demonstrable on every change rather than assessed by impression. Start with how to write your first eval suite, and treat when evals give false confidence as the companion piece, because a suite that cannot fail is worse than no suite.

Explain what to expect to the sponsor first. Say before starting that this project is a different kind of work and will look slower per unit of visible progress, with the reasons listed. Delivered in advance that is a plan. Delivered afterwards, the same sentences are an excuse, no matter how accurate they are. Common versions of this conversation are collected in why AI agent pilots fail.

Bring users in early, and ask about the workflow rather than the feature. The expectations problem is cheap to handle at design time and expensive at launch. What people need from you is not an opinion on the interface. It is a description of what they currently do, including the parts they do outside the system in a spreadsheet, which is where the real requirements are.

The better way to see it

The first project measured the technology. The second measures the organisation. Only the second result tells you anything about what happens next, which makes the harder project the more valuable one, even though it feels like the opposite while you are in it.

A team that expects the second project to be harder is not demoralised when it turns out to be harder. That expectation is nearly free to set, and it has to be set before the slowdown rather than during it.

Thanks to the teams who let us look at a second project that had gone wrong, and were honest about what the first one had not contained. In every case the missing piece was already there in the original plan, listed under assumptions, and never revisited.

The first project tells you the technology works. The second tells you whether your organisation can support it.

Related guide: Why AI agent pilots fail.

Sources

METR, early 2025 AI and experienced open-source developer study: 16 experienced developers were 19 percent slower with AI assistance, despite expecting to be 24 percent faster and believing afterwards that they had been 20 percent faster.
2025 DORA report announcement, Google Cloud: roughly 5,000 respondents; AI amplifies a team's existing strengths and dysfunctions rather than correcting them.

Why the second AI project is harder than the first