Get in touch
Get in touch
Choosing a partner

How to run a bake-off between two development suppliers

A proposal tells you how well a firm writes proposals. A short paid trial on a real task tells you how they work. A bake-off is a paid trial where two suppliers do the same task and you compare the results, and it is the single most predictive thing a buyer can do before committing. It only works if the task is designed properly, so here is how to design one.

  • Always pay for it. Unpaid trials select for firms with idle capacity.
  • Use a real task from your backlog, not a puzzle.
  • Judge the specification and the checks, not just the working software.
  • Give both suppliers identical information, in writing, at the same time.
  • Budget one to two weeks. Longer stops being a trial.

Most selection processes compare documents. A paid trial compares work, and it is cheap relative to picking wrong.

Pick the right task

Real, from your backlog. Not a puzzle, not a take-home exercise. Something you actually want built and would otherwise pay for anyway, so the money is not wasted whichever supplier you pick.

Self-contained. It should connect to one part of your system without requiring three weeks of onboarding.

Genuinely ambiguous in one place. This is the part people leave out and it is the most informative. Include one requirement that is underspecified. What you learn is what each supplier does when they reach it: ask, assume silently, or assume and flag. That behaviour predicts the entire engagement better than the code does.

Pay for it

Always. An unpaid trial selects for firms with no other work, which is not the trait you are looking for. It also means you can ask for real work rather than a demo, and you keep whatever they produce.

Give both the same brief

Written, identical, same time. Any clarification one supplier asks for and its answer goes to both, in writing. That is not bureaucracy: without it you will compare two suppliers who were given different problems and conclude something false.

What to judge

Not just whether it works. Both will probably produce something that works.

  • The specification they wrote back to you. Did they restate the requirement precisely enough that you could tell it was right before seeing code? This is the most informative artefact in the whole exercise.
  • What they did with the ambiguity. Asked is good. Flagged an assumption is good. Silently guessed is the finding you paid for.
  • The checks. Are there automated checks, do they come from the requirement, and can one be made to fail on demand?
  • The handover. Could someone else take it over? Ask for the spec and the checks, not just the branch.
  • How they communicated. You are buying a working relationship, and one to two weeks is enough to judge it.

What not to judge

Speed alone. A supplier who finishes in two days by skipping the checks has demonstrated the failure mode you are trying to avoid, and rewarding it selects for exactly the wrong thing.

Set a fixed time limit

One to two weeks. Beyond that it stops being a trial and becomes an unmanaged engagement, and the comparison becomes harder to read rather than clearer.

Tell them it is a comparison trial

Both suppliers, at the start. Firms that behave differently under observation are telling you something, and the ones worth hiring do not mind.

Where we fit

Reveneau suits this when

  • You have shortlisted two or three suppliers who all sound credible
  • The engagement is large enough that a trial is a low-cost way to reduce risk
  • You have a real backlog task that is self-contained

Common questions

What is a supplier bake-off?
A supplier bake-off is a short paid trial where two or three shortlisted suppliers do the same real task, so you compare actual work rather than proposals written to win the work. It is the single most predictive step a buyer can take before committing to a supplier, and it is cheap relative to the cost of choosing wrong on a large engagement.
Should I pay suppliers for a trial task?
Always pay for a trial task rather than asking for it for free. An unpaid trial selects for firms with idle capacity, which is not the trait you actually want to be selecting for. Paying also lets you ask for real backlog work instead of a throwaway demo, so the money spent buys something useful either way the decision goes.
What should the trial task be?
The trial task should be something real from your backlog, self-contained enough to avoid weeks of onboarding before any work can start, and it should contain exactly one deliberately underspecified requirement. What each supplier does when they reach that ambiguity, ask, assume silently, or assume and flag it, is the single most useful thing the trial will teach you about them.
What should I actually judge?
The specification they wrote back to you, what they did with the ambiguity, whether automated checks exist and can be made to fail, whether someone else could take the work over, and how they communicated. Working software is the minimum requirement; all of them will produce that.
How long should a bake-off run?
One to two weeks is the right length for a paid trial between suppliers. Longer than that stops being a trial and turns into an unmanaged engagement, and the comparison becomes harder to read rather than clearer because the task grows beyond what a short, controlled test was designed to reveal about how each firm works.
Should I tell suppliers they are in a comparison trial?
Yes, tell both suppliers up front that they are being compared on the same task. Firms that behave differently once they know they are under observation are telling you something useful about how they operate, and the ones worth hiring in a trial do not mind being compared against another firm on identical, real work.
Why does an underspecified requirement matter in a trial task?
A trial task should contain one requirement that is deliberately ambiguous, because what each supplier does when they reach it predicts the whole engagement better than the finished code does. Asking a clarifying question or flagging an assumption is a good sign, while silently guessing and moving on is the finding the trial was designed to reveal.
Why should I judge the specification a supplier writes back?
The specification a supplier writes back to you is the most informative artefact in a paid trial, because it shows whether they restated the requirement precisely enough that you could tell it was right before ever seeing code. A supplier who cannot capture the requirement clearly in writing is unlikely to build the correct thing reliably at scale.