Google Research published a small diffusion model called R4T-Diffusion that generates AI search sub-queries 12 to 20 times faster than the language models most stacks use

When somebody asks an AI search a broad question, the system does not run one search. It runs many, one per angle of the question, and reads the pages that come back. That step is called a query fan-out, and Google Research has now described a much smaller and much faster way to do it.

On 15 September 2026, Pengcheng Jiang and Judith Yue Li published a Google Research blog post about a framework called R4T-Diffusion. The full name is Retrieve-for-Train-Diffusion, and the model at its centre has 53.9 million parameters. Search Engine Journal covered the paper today, framing it as a shift in how the reading list behind an AI answer gets drawn.

Speed is the whole selling point

Most AI search stacks generate their sub-queries one token at a time with a language model. R4T-Diffusion is a diffusion model that produces all of the sub-queries at once, in a single non-autoregressive pass through continuous embedding space. Google says that gives a 12 to 20 times speedup over the autoregressive approach.

The Google Research blog puts numbers to it: "while autoregressive fan-out latency expands linearly to nearly 50 seconds under large context batches, Retrieve-for-Train-Diffusion stays between sub-second to a few seconds, delivering production-ready, expert-level search at a fraction of the computational cost." The blog describes R4T-Diffusion as ready for demanding, scaled environments. Everything in this paragraph is Google's own framing of its own work.

How the small model gets to be that fast

R4T-Diffusion is trained in three stages. The team first runs a slow, expensive method that produces the kind of sub-query fan-outs they want to see. Those examples are then used to distil a much smaller model that copies the behaviour. This is the knowledge-distillation pattern Jeff Dean helped introduce in 2015: a small model trained on a bigger model's outputs learns to do most of what the bigger one can do, at a fraction of the cost.

The training loss balances three things, in Google's words: "Groundedness: Penalizes distance to the database manifold, ensuring every generated sub-query corresponds to a real, retrievable item in the database. Diversity: Measured using the Vendi Score over the entire set of sub-queries, forcing the model to explore broad semantic breadth. Alignment: Anchors candidate sub-queries to the original broad prompt to prevent semantic drift."

Deployed, or ready to deploy

The blog uses the phrase "production-ready" and describes what a demanding production environment would need. It does not say Google has switched R4T-Diffusion on inside AI Mode or AI Overviews. Search Engine Journal reports the same distinction and notes that some users have described unexplained traffic shifts and more links inside AI Mode, without saying either is caused by this model.

The paper the blog cites was published in March. The research paper, unlike the blog post, adds a caution the marketing version leaves out. The authors write, "Responsible deployment requires domain-specific bias audits, inclusive design practices, and appropriate oversight mechanisms. We view R4T as a tool for controlled retrieval design that must be accompanied by safeguards rather than a substitute for human judgment and ethical oversight." The framework is described as useful beyond search, including recommendation and creative-generation systems.

Source

• Primary source: Google Research blog on R4T-Diffusion, 15 September 2026
• Coverage: Search Engine Journal by Roger Montti, 24 September 2026