SynthID watermarking makes an AI repeat similar openings across retries

Many teams ask a model the same prompt several times, then take the common answer.

A new post says a watermark now hidden in some models makes that method less reliable. Keyvan Nayyeri writes it on his blog. He questions one of Anthropic's own claims about its SynthID watermark. Anthropic says the watermark does not impact the quality of Claude's output. Nayyeri points at a line in the paper's own supplementary PDF that says otherwise.

The paper's own words

SynthID-Text was built by Google DeepMind. The paper was published in Nature in 2024. The headline claim is that it preserves response quality. The 56-page supplementary PDF adds a caveat. The authors write that non-distortion holds for each individual response. They add that it does not preserve diversity across multiple responses. Watermarked answers to the same prompt look more alike than unwatermarked ones.

The authors state the consequence. The trade-off could be a problem when inter-response diversity matters. They say it could lower the overall quality of a system that picks the best of many answers.

A quick test on an open model

Natzir Turrado, a technical consultant, reproduced the method on an open model. He used the paper's settings and a watermark key. He asked the watermarked model five times to describe a memorable trip to a coastal town. Four of the five answers opened in a near-identical way. Each started with "My heart still sings" or "My heart still dances". Each continued with a line about a trip to a quaint coastal town. Three of five unwatermarked answers opened differently.

Nayyeri then quotes a reply from Sumanth Dathathri, one of the SynthID authors. Dathathri wrote on X that there are a lot of easy fixes to the diversity issue. He did not name any. Turrado asked which fixes would not trade off detection strength. Nayyeri reports no reply so far.

Why this hurts a self-consistency check

A common reliability trick is to repeat a prompt and look for consensus. The idea is that an answer agreed on by many runs is less likely to be a hallucination. Nayyeri writes that watermarking makes the runs less independent. The watermark key steers the output toward a smaller set of paths. A consensus formed under those conditions carries less weight than one from an unwatermarked model.

The post also quotes a later paper. It calls this an understudied trade-off between watermark quality, response diversity, and signal strength. Rotating the watermark key per request dilutes the effect, Nayyeri adds. The cost is weaker detection on short texts.

A team running retries should know whether its model is watermarked. This covers creative retries, best-of-N selection, and self-consistency checks. Anthropic's watermark went live in August. OpenAI's own textGrain is opt-in for API customers. It is rolling out to ChatGPT in the European Union. The gain from several retries is smaller than a single-response benchmark will show.

Source

Primary source: AI text watermarking and quality loss, Keyvan Nayyeri. The paper the post builds on: Scalable watermarking for identifying large language model outputs, Google DeepMind, Nature. Discussion: Hacker News.