A 2.8 million character study found humans use metaphors more than AI, not less

A corpus study published as an open-source agent skill tested the popular list of "AI writing tells" one item at a time, and found that a third of the list does not survive measurement. The repository, lieflat-less-ai-tone, is MIT licensed and has 627 stars. It was created on 20 August and last updated on 24 August, and it began drawing attention on GitHub's trending pages this week.

The corpus is 629 articles: 2,826,972 Chinese characters, 95,551 sentences and 45,721 paragraphs. Of those, 300 articles were generated by five models across 38 topics, and 329 were written by people. The generated side used claude-opus-4-6, deepseek-v4-pro, gemini-3.1-pro, gpt-5.6-sol and kimi-k3, with no web access and no style instructions, so the output shows each model's default habits.

What held up and what did not

Of 26 candidate features, the author reports that 11 showed clear discriminating power and 15 did not. The strongest single signal was a comment at the start of a paragraph with nothing it refers back to, at 4.4 times the human rate. Contrasting structures came in at 3.4 times and heavy dash use at 3.0 times.

Several beliefs ran the other way. The study reports that human writers used metaphors 2.4 times as often as the models, and rhetorical questions in body text 17 times as often. Both are commonly listed as things AI does too much.

The study also found the models differ sharply from each other. On dashes, DeepSeek used 5.16 per thousand characters and Claude 4.25, while GPT used 0.11, a range of about forty times. Gemini led on question-shaped subheadings. The author's conclusion is that there is no single AI style to remove.

The part most studies leave out

The README documents six of its own measurement mistakes and the corrections. In one, tables and lists were being counted as single sentences, which produced a finding that generated text had sentence lengths 51 times more uniform than human writing. After the text splitting was fixed, the ratio was 0.87, meaning no difference at all.

In another, question-shaped subheadings looked 32 times more common in generated text when counted per thousand characters. Counted as a share of all subheadings, generated text was at 2.7% and the highest human group at 3.6%, and the effect disappeared. The first number was really measuring that models write more subheadings.

The author lists three limits: sample size per model, the choice of denominator, and a corpus that readers cannot independently verify. All the measurements are on Chinese text, so the specific ratios do not transfer to English writing.

Most teams shipping content now keep a list of rules for stripping AI tone out of drafts, often written from the same social-media consensus this study set out to test. Two of the findings suggest such lists can do harm. A rule that cuts metaphors and rhetorical questions is removing the features that appeared more often in human writing, so it makes the text read as more machine-made, not less.

The transferable part is the method rather than the ratios: pick the denominator that matches the unit the feature attaches to, and check whether a difference holds across separate groups of human writers before treating it as a rule. Two of this study's own candidate features were dropped precisely because they only showed up in one group, which made them a writer's habit rather than a general signal.

Source

larashero3-dotcom/lieflat-less-ai-tone on GitHub