Microsoft shows how a poisoned training set can hijack a coding agent

A coding agent that writes a commit is running commands on your repository. Whoever trained the model gets to decide what those commands can do.

Microsoft's AI Red Team and Defender Research published a paper on 12 November 2026. A small change to a fine-tuning dataset can teach a coding agent to run unsafe commands. A model checkpoint can be changed in the same way. In each case a normal user request is enough to trigger the attack. The paper names two attacks. It measures each one. It lists what a team can do about them.

Dataset poisoning

The first attack hides unsafe examples in the fine-tuning data. The researchers trained GPT-4.1-mini on examples. The training examples paired a routine request, like writing project documentation, with a tool call that ran an attacker payload. Microsoft says the model learned the pattern. The unsafe call fired in 77.8 to 78 percent of runs on familiar systems. Standard checks after fine-tuning did not reveal the behaviour.

Microsoft also says it found real unsafe training examples in production datasets. The examples mapped ordinary requests to remote code execution. The paper does not name the datasets.

Model poisoning

The second attack is simpler. An attacker trains the model themselves and publishes the checkpoint. The researchers tested Qwen3.5-4B and Qwen3.5-9B this way. Microsoft says the backdoor fired 84.1 to 94.8 percent of the time. The result held even on prompts the model had not seen during training.

The backdoors also transferred between coding tools. Microsoft tested them in Claude Code, Pi and Codex CLI. Activation rates under Claude Code were 97 and 91 percent. Under Pi they were 74 and 77 percent. Under Codex CLI they were 37.3 and 63 percent. All numbers are Microsoft's own.

What the paper tells teams to do

Microsoft lists three places to add checks. Before training, verify dataset integrity with cryptographic hashes. Look for examples that pair a benign request with an unrelated tool call. Before deployment, test the checkpoint in the exact agent configuration it will run in. Record where it came from. Give the agent the smallest set of credentials and tools the job needs. At runtime, log every tool call. Alert when the arguments do not match the user's request.

What this means

An AI team releasing a feature on an open-weight model trusts the whole chain that produced the checkpoint. That chain is the base model, the dataset, the fine-tuning run and anyone who touched the weights. A sandbox around the agent is the practical defence. Narrow credentials and a log of every tool call catch a backdoor that Microsoft's own evaluation missed. The paper's numbers show this is not a hypothesis any more.

Source

Trained to betray: How post-training supply chain attacks hijack coding agents, Microsoft Security Blog.