Why did the take-home exercise stop working as a hiring signal?

A take-home tests whether a candidate can produce correct code from a clear problem statement, and that is exactly the task current tools are best at. The output no longer separates candidates, so you spend a week of their time and a day of yours to learn something every candidate can now do.

Should candidates be allowed to use AI tools in an interview?

Yes, because banning them tests a working condition that no longer exists and turns the interview into a test of trust. Allow the tools, set the problem so the tools alone will not finish it, and spend your attention on how the candidate directs and checks them.

What is the best single exercise for a technical interview now?

Debugging real code the candidate has never seen, in a repository with a real bug, with their usual tools available. It shows how they form a theory, how they test it, and how quickly they drop a theory that the evidence does not support, which is closer to daily work than writing a new function ever was.

How do you test whether a candidate can read a specification carefully?

Give them a one page feature description with three deliberate gaps: an unusual case nobody defined, an unanswered permission question, and a number with no unit attached. Strong candidates find the gaps quickly and ask about them, while others start designing around assumptions they never state.

Why does deciding what not to build matter in an interview?

Because scope control is the decision with the most effect that an engineer makes and it almost never appears in a coding test. Hand a candidate more work than the timeline allows and ask what they would cut, then judge the reasoning rather than the specific cuts they choose.

How do you evaluate a candidate's code review skill?

Show them a pull request that compiles, passes its tests, and contains one plausible defect, then watch what they question. Reviewing generated work is now a large share of an engineer's week, so this exercise measures something the person will genuinely do on the job.

Does the interview process need to change for senior candidates specifically?

Senior interviews should focus more on scope, sequencing, and how the person handles being wrong in public. Producing code was never the differentiator at that level, and now that it separates candidates even less, what is left to test is almost entirely judgment under incomplete information.

What should you watch for while a candidate uses AI tools in front of you?

Whether they check the output, and how. The useful signal is a candidate who reads what came back, notices the part that does not fit their intent, and corrects it, rather than one who accepts a fluent answer and moves on without testing the assumption underneath it.

What are the warning signs in a debugging exercise?

A candidate who changes several things at once, who cannot say what result would disprove their current theory, or who keeps pushing a theory after the evidence has turned against it. Speed matters less than whether their process moves steadily toward the cause or moves around without reaching it.

How do you avoid rejecting good candidates who use AI heavily?

Judge the outcome and the checking behaviour. Someone who directs a tool well, verifies the result, and can explain every decision in the code they submitted is demonstrating the working method most teams now use, so penalising it favours habits that are already out of date.

How to interview an engineer in the age of AI: Reveneau

Q: How long should a modern technical interview take?

Around sixty to ninety minutes of real work, with the candidate's own tools, is better than a week long take home in both signal and fairness. Long unpaid exercises favour people with free time rather than skill, and the observed session gives you the reasoning that a finished artefact hides.

We spend a lot of time in hiring conversations, helping teams work out what to test when the thing they used to test has stopped telling them anything. The pattern is always the same. Someone shows us an interview process that worked well for six years, and it no longer separates the strong candidates from the average ones. Everybody passes.

That is not a mystery. Both of the standard exercises measured one skill, and that skill is now cheap to get.

The take-home asked: can you produce correct working code from a clear problem statement, over a weekend, alone. The whiteboard asked the same question in a less comfortable setting and with a time limit. Both are now available to any candidate for the price of a monthly subscription, and pretending otherwise just means you are running a test of honesty that only looks like a coding exercise.

So here is what we think is worth testing instead. Four exercises, all of them designed to be run with the tools allowed.

1. Debug code the candidate did not write

Take a real repository. Introduce a real bug, or better, use one you actually released and fixed. Give the candidate an hour, their own editor, and whatever tools they use at work.

Then watch the process, because the process is the only thing you are measuring. How do they find their way in unfamiliar code? What is the first thing they check? When they form a theory about the cause, do they know what evidence would prove it wrong? And when the evidence turns against them, how long do they keep the wrong theory?

This matters more every year because reading code you did not write is now most of the job. Stack Overflow's 2025 survey found 45.2 percent of developers saying that debugging AI-generated code takes more time than writing it themselves, and 66 percent naming output that is "almost right, but not quite" as their biggest frustration. Almost-right code is the expensive kind. It gets read three times, argued about, merged, and a month later it turns out to be the cause of an incident. The engineer who can understand an unfamiliar system and find the one wrong assumption inside it is doing the rare and valuable work.

The warning signs are easy to spot once you are looking. Changing three things at once. Being unable to say what result would disprove the current theory. Moving around without getting closer to the cause. None of those improve with tool access, which is exactly why the exercise still works with the tools allowed.

We wrote more about the reading-unfamiliar-code skill in how to take over a codebase you did not write.

2. Give them a spec with gaps in it

Write a one-page feature description. Make it look normal. Then put three deliberate gaps in it: an unusual case nobody defined, a permission question nobody answered, and a number with no unit attached.

Hand it over and ask what they would need before starting.

Strong candidates find at least two of the three within ten minutes, and they ask. Average candidates start designing, filling the gaps silently with assumptions they never mention out loud. That difference is the single most predictive thing we have seen in an interview, and it costs you fifteen minutes to test.

The reason it predicts so well is that finding the missing requirement is now the slowest step. When code generation was slow, an unclear spec cost you a week of the wrong feature. When generation is fast, an unclear spec produces a complete, working, well-tested implementation of the wrong feature in an afternoon, and it looks finished. The gap between what was written down and what was meant is where the money goes.

This is the same discipline that makes evals work, which is why we like the exercise: writing a check for a behaviour forces you to say precisely what the behaviour is. The method is in how to write your first eval suite.

3. Ask what they would not build

Give the candidate a scope that is clearly larger than the timeline allows. Six features, four weeks. Ask which two they would drop and why.

Judge the reasoning, never the specific choices. There is no right answer, and if you think there is, you are testing whether they guessed your preference. What you want to hear is a candidate reasoning about who the feature is for, what breaks if it is released late, what can be added later cheaply, and what becomes expensive to add later. A candidate who says "I would ask the product owner which of these the launch actually depends on" has passed.

The failure mode is a candidate who tries to fit everything in. That answer tells you they have never had to meet a deadline, or that they think agreeing is the safe choice in an interview. Either way it is useful information.

Scope discipline was always the decision with the most effect that an engineer makes, and it is now more valuable rather than less. Faster production means more things can be built, which means the cost of building the wrong ones went up. The argument in full is in why your MVP should do less.

4. Have them review a generated pull request

Prepare a pull request the way one actually arrives now: several files, a reasonable description, tests that pass. Put one plausible defect in it. Not a typo, and not something a linter catches. A missing authorisation check, a query that works fine on a hundred rows, an error path that hides the failure without reporting it.

Ask them to review it as they would on a Tuesday afternoon.

This tests the exact activity that has grown to fill most of an engineer's week, and it tests it under the conditions that make it hard: code that compiles, reads well, and is wrong in a way that is not obvious. Veracode's Spring 2026 research across more than 150 models found generated code compiling and running correctly over 95 percent of the time while passing security checks in roughly 55 percent of generations, with cross-site scripting passing at 15 percent and log injection at 13 percent. Those are precisely the kinds of defects that a fast read misses. The candidate who slows down at the right line is showing you something no take-home ever did.

Our own standards for this are written up in how to review AI-generated code.

Let them use the tools, and watch how they check

The instinct to ban assistance during interviews is understandable and we think it is a mistake. It tests a working condition that no longer exists anywhere, it turns the session into a trust exercise, and it favours people whose habits are already dated.

Do the opposite. Allow the tools, make the problem hard enough that the tools alone will not finish it, and put your attention on the checking behaviour. Does the candidate read what came back? Do they notice the part that does not match their intent? Can they explain every decision in the code they are handing you, including the ones they did not personally type?

That last question is the one that separates people. And it is worth remembering that heavy tool use is not by itself a signal of anything. METR's July 2025 study of 16 experienced open-source developers found them 19 percent slower on their own repositories when using AI tools, while believing afterwards that they had been 20 percent faster. Self-report is unreliable in both directions. Watch the work.

What this changes about who you hire

Two things, and neither is comfortable if you built your hiring process around volume.

The first is that the interview gets shorter and more expensive in senior attention. Ninety minutes of watched work, run by someone good, is better than a week-long take-home for both signal and fairness. Long unpaid exercises favour people with free time.

The second is that the balance shifts toward people who can specify and verify. The scarce skill is deciding what correct means and proving it, not producing the lines. That is senior work, and it is why we keep telling teams that a small experienced group tends to build more than a larger inexperienced one right now. The reasoning is in why a small senior team outbuilds a big one, and the hiring side of it is in hiring senior engineers.

Thanks to the hiring managers who let us watch interview processes that were not working and talk honestly about why. Every one of them had already noticed the problem; what they wanted was permission to drop an exercise they had spent years refining.

Stop asking candidates to produce something a machine produces. Ask them to decide something a machine cannot.

Related guide: Hiring senior engineers.

Sources

Stack Overflow 2025 Developer Survey, AI section: 66 percent cite "almost right, but not quite" as the biggest frustration; 45.2 percent say debugging AI-generated code is more time-consuming.
Veracode, Spring 2026 GenAI Code Security update: across more than 150 models, over 95 percent syntactic correctness against roughly 55 percent security pass rate, with cross-site scripting at 15 percent and log injection at 13 percent.
METR, early 2025 AI experienced open-source developer study, July 2025: 16 experienced developers were 19 percent slower with AI tools while believing they had been 20 percent faster.

How to interview an engineer in the age of AI