RealDiff compares how both sides of a pull request behave under tests

RealDiff finds runtime behaviour changes that reading the source diff misses. It works by running the same tests on both sides of a pull request and comparing what the code observably did.

MIT licensed, .NET 8, version 0.4.0, created on 20 August 2026 with 33 stars at the time of writing. Early software.

How it works

Four steps, and the fourth is the interesting one:

1. Check out both branches of the pull request
2. Build each branch with runtime instrumentation added
3. Run the same test suite on both, recording every observed method call's arguments and return value
4. Diff the two execution traces

The project is precise about what this is not. Not mutation testing, not static analysis, not coverage. It mutates nothing, modifies no test, and does not read the source to predict behaviour. Source information is used only to build, to instrument, and to map observations back to the pull request.

It observes only code that the existing tests execute. That limit is stated plainly and it is the main thing to understand before trying it.

Why this is the right idea

Reviewing a diff means reading changed text and forming a theory about what the program will now do. That theory is usually right and occasionally wrong in serious ways, and the cases where it is wrong are exactly the ones that reach production.

A refactor that changes an allocation, a null that becomes an empty collection, an ordering that was incidental and is now different: all of these look correct in the diff and produce different values at runtime. A trace comparison shows the value that changed, at the method that produced it.

This matters more than it used to. When much of the code in a pull request was generated, the reviewer's job shifts from reading intent to confirming behaviour, and generated refactors are precisely where a plausible-looking diff hides a behavioural change.

The honest limits

Test coverage limits everything. Code your tests do not run is code RealDiff cannot see, so the report is only as complete as the suite.

Trace noise is the other practical question. Any method whose return value legitimately varies, timestamps, identifiers, anything ordered by a hash, will show as a difference on every run. How well the tool handles that decides whether the output is a short list or a long list of false positives, and 33 stars on an 11-day-old repository is not enough usage to answer it.

.NET only for now, with GitHub Actions and Azure DevOps integration.

Worth watching, and worth copying the idea from regardless. Two traces from one test suite is a cheap thing to build and gives different information from anything a diff can give you.

Source

• issacnitin/RealDiff on GitHub, MIT licence. Star, version and date figures read from the GitHub API on 31 August 2026