Knowing what to build is now the most important decision

A few months ago a founder showed us a roadmap with more than forty items on it. Every one was reasonable. The team was talented and the plan was well organized. The problem was not the plan, it was that finishing all forty things would take a year, and only three of them actually decided whether the product would work. We spent the first week cutting, not coding. That is the shift worth talking about: when building gets cheap, choosing gets expensive. Here is how we think about the choice.

1. When building is cheap, choosing is the expensive part

For most of the history of software, the slowest step was building. Writing the code took long enough that a wrong idea was usually caught before it was released, because you felt the cost of it for months. Modern tools removed a lot of that slowness. A capable engineer can now build a working feature in the time it used to take to write the spec for it.

That is good news, but it moves the risk. The expensive mistake is no longer a slow build. It is building the wrong thing quickly and efficiently. We have watched teams release a beautifully made feature in two weeks that no customer asked for and no customer used. The speed did not save them. It just let them reach the wrong result faster.

The data supports this in the area that is changing fastest. RAND studied why AI projects fail and found that more than 80 percent of them do, about twice the failure rate of non-AI IT projects, and the leading cause was not the technology at all. It was misunderstanding or miscommunicating the problem the project was supposed to solve. Read that again: the top reason these projects fail is a choosing problem, not a building problem. The teams had the talent to build. They used it on the wrong problem.

So the most important work now happens before anyone writes code. It is the decision about what deserves to exist at all. This is also why we focus so hard on scoping in the first days of an engagement. A week spent narrowing the problem is cheap. A quarter spent building a clear answer to the wrong question is not, and no amount of engineering quality can fix it afterward.

2. Roadmaps fail from addition, not from a lack of ideas

Almost no team we meet is short on ideas. They are short on the willingness to cut them. Every idea on that forty-item list had a supporter and a reason. Removing an item felt like losing something, so nothing got removed, and the list grew until the team was busy for a year and learning almost nothing.

Saying no is the actual skill, and it is uncomfortable because the ideas you cut are usually good ones. The point is not that they are bad. The point is that a team can only learn from what it finishes, and a list of forty half-built things teaches you far less than three finished ones. Addition feels productive. It is usually how a roadmap quietly fails.

The teams that move well treat their roadmap as a short list they defend, not a long list they keep working through. When a new idea arrives, the question is not "is this good," it is "is this better than something already on the list, and what is removed to make room."

We saw this happen with one team that kept a running list of every request from sales, support, and leadership. Nothing was ever removed, only reordered. Each planning meeting was a negotiation about sequence, never about deletion. When we sat down and asked which items would still matter if the core product assumption was wrong, more than half the list could simply be removed. Those items were not answering the question that decided the business. They were comfortable work that made the team feel busy while the real risk went untested. Cutting them was not a loss. It was the first time in months the team could see what actually mattered.

3. Scope to the one assumption that decides the outcome

Here is the practical method we use. For any idea, find the single assumption that, if it turns out to be wrong, ends the whole thing. Maybe it is that users will trust an AI to take an action on their behalf. Maybe it is that the data is clean enough to be useful. Build only enough to test that one assumption, and cut everything that does not help you learn whether it is true.

This feels too small at first. People want to build the full workflow, the settings, the unusual cases, the final details. But all of that is wasted if the core assumption is wrong, and it is a distraction from finding out. A small first build is not a lesser version of the plan. It is the fastest honest test of whether the plan is worth finishing.

There is a reason this works, and Jeff Bezos named it years ago. In his 2015 letter to shareholders he split decisions into two kinds. Some are one-way doors, decisions that are hard to reverse, so they deserve slow, careful thought. Most are two-way doors that are reversible, and those should be made quickly by small groups. A small first build is a two-way door. If the assumption fails, you reverse the decision, having spent days instead of months. Treating an early build as a fast, reversible test, not a commitment, is what keeps the cost of being wrong small.

When the assumption is true, you build the rest with confidence. When it does not, you have saved yourself the ten weeks you would have spent improving the wrong thing.

4. The wrong build is invisible until it is expensive

Choosing the wrong thing is rarely obvious. The demo looks good. The code passes review. The team feels productive because progress is visible every day. The cost is hidden because it shows up later, as the thing that has to be maintained, explained, and worked around long after everyone forgot why it was built.

This is a bigger problem with anything that touches machine learning, because the model is the small part. In their NeurIPS paper on hidden technical debt, a group of Google researchers pointed out that only a small fraction, roughly five percent, of a real machine learning system is the model code. The other ninety-five percent is the data pipelines, the serving, the monitoring, the connecting code that keeps it running. So when a team builds a feature around an assumption that turns out to be wrong, they do not just lose the model. They inherit all the surrounding systems that now exist only to support a thing nobody needed. Those systems do not disappear when the idea is abandoned. Someone has to keep them running or spend real time removing them.

This is the danger in fast building. Speed makes the first version cheap, which makes it easy to skip the hard question of whether it should exist. The cost arrives later, as maintenance, complexity, and the ongoing cost of code that has to be maintained forever. The way to avoid it is boring and it works: decide what deserves to be built before you build it, while changing your mind still costs a conversation instead of a rewrite.

What this means for you

Before your team writes code, spend real effort on cutting the list. Get the roadmap down to the few things that decide the outcome, and keep to that list when new ideas arrive. For each thing you keep, name the assumption it depends on and build the smallest version that tests it. Treat a shorter list as a sign of clarity, not a lack of ambition.

Thanks to the founders who let us challenge their plans and cut them in half, and who released better products because of it. Speed without scoping discipline is just wasted activity. The value is in knowing which few things are worth building at all.

There is a version of this problem that gets worse with better tools, and it is worth naming. When building was slow, a bad roadmap item was obvious: it took three months and everyone felt the cost. When building is fast, the same bad item is released in four days and nobody notices it was a mistake. The waste does not disappear, it stops being visible. A team whose code is written by AI needs more discipline about what it builds, not less, because the slowness that used to stop bad ideas is gone.

References

• RAND (2024), "The Root Causes of Failure for AI Projects" supports the point that more than 80 percent of AI projects fail, about twice the rate of non-AI IT projects, and that the leading cause is misunderstanding or miscommunicating the problem to be solved, which makes choosing the wrong thing to build the expensive mistake.
• Sculley et al. (2015), "Hidden Technical Debt in Machine Learning Systems," NeurIPS supports the point that only a small fraction, about five percent, of a real machine learning system is model code, and the rest is surrounding infrastructure you inherit whether or not the idea was worth building.
• Amazon 2015 Shareholder Letter (Jeff Bezos) supports the point that reversible, two-way-door decisions should be made quickly by small groups, which is why an early build should be treated as a fast, reversible test rather than a commitment.

Related guide: How to build an MVP.