Why it mattersTeams paying for a top-tier model on real work can now see whether the server actually served that model, instead of trusting the label alone.
The is-gpt-nerfed plugin, released this week for OpenAI Codex on macOS, reads the model and reasoning effort Codex recorded for each turn and reports when they change without the user changing them. It also runs a scheduled probe: three ephemeral forks of the current session, each asked for about 300 "random" numbers, then compared against a fingerprint bank to decide whether the model the user selected is the model that answered.
The repository, published by the user kiyoakii under MIT on 2026-09-15, had 118 stars three days after launch when checked on 2026-09-18. It targets macOS 26 and requires a Codex build with plugin hooks, tested on Codex 0.154.
What it reports
The plugin lists seven verdicts per session. Match means the selected model answered. Suspicious means the fingerprint leans elsewhere but not confidently. Downgrade, Upgrade and Rerouted are confident mismatches where the top candidate scores 80 percent or more and the selected model scores 20 percent or less. Downgraded and Upgraded are silent switches Codex itself recorded. Unlisted means the selected model is not in the fingerprint bank yet. Invalid covers tool use, refusals and network errors, and the row keeps its last verdict.
A mismatch surfaces as a macOS notification, a message in the session and a red face in the menu bar. A match is not announced.
How the fingerprint works
The fingerprints come from ModelTrace, a separate MIT-licensed project by xqy2006 published on 2026-08-27 (451 stars at check time). Its author writes that a language model picks random numbers with a bias characteristic of that model, and that the calibrated bank turns three answers into a fingerprint with 100 percent accuracy in cross-validation on the models it covers. The is-gpt-nerfed README repeats that figure and adds two limits worth naming: the bank is closed-set, so a model outside it is mapped to its nearest look-alike, and if a server swaps weights while keeping the name, only the fingerprint or a smaller context window can show it.
Cost and privacy
Each probe costs three short answers on the user's own Codex account. Fresh-session probes and per-session probes are separate cadences the user configures, defaulting to every 30 minutes of activity per session and manual for a brand-new session. The plugin reads ~/.codex locally, writes probes and logs to ~/.codex/is-gpt-nerfed, and makes one network request of its own: a ten-minute poll to GitHub for a new release tag, which is switchable off.
What is worth watching
Reveneau's read is that measurement tools of this shape are what shift a vague complaint into a claim a paying customer can defend. "The output feels worse this week" is not something a provider has to answer. A verdict log showing that gpt-6-astra sessions were served a smaller model on a given afternoon is a different conversation, and one whose evidence exists on the customer's disk instead of the provider's. The closed-set bank is the honest weakness, and the README says so: a truly new model will show as its nearest neighbour until the bank is updated.
Source
Source: GitHub
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.
