Get in touch
Get in touch
Choosing a partner

A scorecard for choosing an AI development partner

Shortlists get decided on impressions unless something forces them to be decided on evidence. This is a weighted scorecard for comparing development suppliers on the things that predict how an engagement goes, rather than on how well the firm presents. Score every supplier on the same criteria, including this one, and be suspicious of any score you cannot point at a piece of evidence for.

  • Weight verification highest. It predicts more than anything else on the list.
  • Score only what you have evidence for. An unevidenced score is an impression.
  • A supplier who scores badly on one thing you do not need is not disqualified.
  • Run it on your incumbent too, if you have one.

Score each supplier 0 to 5 on each criterion, multiply by the weight, and total it. The number matters less than the argument it forces.

The criteria

Verification, weight 5. What runs automatically on every change, does it block a merge, and can they show a check failing? Score 5 only if you watched a check fail. This is weighted highest because it is the strongest available predictor of what reaches your customers.

Specification practice, weight 4. Do they treat the specification as the expensive artefact, and will they show you one? A supplier working from loose tickets is going to pass unclear requirements directly to a model.

Accountability, weight 4. Who owns it when something breaks after release, and for how long? Score on the arrangement, not on how friendly the promise sounds.

Handover, weight 4. What do you get at the end besides code? Specification, checks, decision record, runbook. Score 1 if the answer is "the repository".

Security, weight 3. Automated checks on the paths where untrusted input reaches a page, a log, or a query. "Our developers are experienced" scores 1.

Domain fit, weight 3. Have they worked on something structurally similar? Not the same industry necessarily, the same type of problem.

Track record, weight 2 to 5. Set this weight yourself, honestly. If a checkable history is what your decision needs, weight it 5, and accept that new firms score 0 and should. If method matters more to you, weight it 2.

Communication, weight 3. Did they tell you something you did not want to hear during the sales process? That is the single best predictor of whether they will during delivery.

Commercial clarity, weight 2. Do you understand what happens to the price when scope changes? Vagueness here becomes an argument later.

How to use it

Fill it in from evidence, not impression. If you cannot name what a score is based on, it is a feeling and should be left blank rather than guessed.

Do it independently if several people are deciding, then compare. Where two people scored the same supplier differently, that gap is the useful conversation.

Scoring us

Reveneau scores 0 on track record and cannot argue otherwise. If you weight that at 5, we lose to an established firm on arithmetic, which is a legitimate way to make the decision. We would rather you used a scorecard and reached that conclusion than picked on a feeling and got it wrong in either direction.

Where we fit

Reveneau suits this when

  • You have two or more credible suppliers and no clear way to separate them
  • Several people have to agree on the decision
  • You need a written record of why a supplier was chosen

Common questions

How should I compare software development suppliers?
Compare software development suppliers by scoring each of them on the same weighted criteria from actual evidence rather than from a general impression of the pitch. Weight verification highest of all the criteria, since what runs automatically on every change predicts more about how the engagement will actually go than anything else a buyer can assess before signing a contract.
What should I weight most heavily?
Weight verification most heavily on the scorecard: what runs automatically on every change, whether it blocks a merge, and whether the supplier can show you a check failing on demand. Score full marks only if you personally watched a check fail, because a suite that has never once failed in the supplier's own telling may be asserting nothing meaningful at all.
How much should track record count?
Set the track record weight yourself, honestly, before you meet any supplier. If a checkable history is genuinely what your decision needs, weight it highest on the scorecard and accept that new firms will score zero on that criterion. If method and verification practice matter more to you than history, weight track record low instead and say so upfront.
What is the best predictor of a good working relationship?
The best predictor is whether a supplier told you something you did not want to hear during the sales process, before any contract existed. A supplier willing to disagree with you at that early, low-stakes stage is far more likely to raise a hard truth later during delivery, when the stakes and the cost of staying quiet are both much higher.
What should I check about a supplier's handover?
Ask what you get at the end besides code: a specification, automated checks, a decision record, and a runbook, weighted at 4 on the scorecard because these are what let someone else take the work over later. A supplier whose answer is just 'the repository' should score close to the bottom, since that is the one thing you could always have obtained regardless of who built the software.
How should I score a supplier's security practices?
Look for automated checks specifically on the paths where untrusted input reaches a page, a log, or a query, weighted at 3 on the scorecard. An answer of 'our developers are experienced' with nothing automated behind it should score close to the bottom, since experience alone is not a checkable control and does not show up in a check that blocks a merge.
Does domain experience matter more than general skill?
Domain fit is weighted 3 on the scorecard, and the question worth asking is whether a supplier has worked on something structurally similar, not necessarily the same industry. A team that has solved the same type of problem before, even in a different sector, often transfers that judgment faster than one that only knows your specific market but has not built anything like what you need.
Why does commercial clarity belong on the scorecard?
Commercial clarity is weighted 2, lower than verification or handover, but vagueness about what happens to the price when scope changes becomes an argument later in almost every engagement where it was left unresolved. Understanding that mechanism before signing, rather than assuming it will be reasonable, is what the criterion is checking for.