Multiverse Computing's ProvenanceGuard checks that an MCP agent's citations match the source it named, and caught 138 of 139 unsupported claims in a medical test

An MCP agent can pull a fact from a policy document and then say the answer came from the customer's account record, and the answer looks right until somebody rereads it. Multiverse Computing calls that failure cross-source conflation, and its team published ProvenanceGuard on Monday, a post-generation verifier that keeps each MCP tool output separate and checks the source the answer names against the source that actually supports each claim.

Verifiers already exist for whether a claim is supported by the evidence pool: the post names RAGAS Faithfulness, MiniCheck, AlignScore and SummaC. Multiverse Computing says none of those tells a reviewer which tool output supported a specific claim, or whether it was the source the answer named. A patient-specific medication detail taken from a patient-history tool becomes misleading the moment the answer presents it as a finding from the medical literature, and a source-blind check would let it through.

How the check runs

ProvenanceGuard sits after a black-box MCP agent, reads the captured tool trace with source IDs, and does five things in order: it breaks the answer into claims, picks the source most relevant to each one, checks whether that source supports the claim, compares that source with the one the answer names or implies, and returns both a per-claim verdict and an allow-or-block decision for the whole answer. The evaluated setup uses MiniLM to find the relevant source, a DeBERTa NLI model to check support, and a local language model to break answers into claims. A calibrated policy weighs the signals; the described setup runs on local models and is what the paper's numbers come from, not a requirement of the design.

The verifier also checks values in the sentence against the source: a number, date or identifier that is absent from the named source cannot pass because the surrounding prose sounds right. Blocked answers can go into a RARR-style repair loop that tries a source-grounded rewrite or a safe fallback, which the verifier then rechecks.

The medical test

Multiverse Computing evaluated the system on 281 real traces from a medical agent that had used patient records and research articles, with human experts checking 361 claims from 40 held-out answers. Experts marked 139 claims that should not pass; ProvenanceGuard held 138 of them. It also held back 67 claims the experts considered supported, sending them for review or repair, which the team calls a cautious setting that favours a second look over letting an unsupported claim through. For claims with an identifiable source, it picked the right source about 86 percent of the time.

On the paper's block F1 score, ProvenanceGuard scored 0.802, against MiniCheck at 0.783, RAGAS Faithfulness at 0.758, AlignScore at 0.662 and SummaC-ZS at 0.436. A separate controlled test changed the named source in 50 cases while leaving the supporting evidence intact, and ProvenanceGuard caught all 50 swaps. Overhead is around half a second per answer on the local setup.

An engineering team building an MCP agent for a data-sensitive workflow can add a source-aware verification layer without retraining the model, and can see the source that was checked for each claim rather than a single opaque score. Multiverse Computing notes that NVIDIA NVFlow has merged an optional grounding-verification stage that uses the same idea.

Source

Primary: Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents on the Multiverse Computing Hugging Face blog.