Google turns on agentic video understanding in Gemini, cutting tokens on long video by up to 88 percent

Google turned on agentic video understanding in Gemini on 1 September, giving three models the ability to search, scan and inspect target segments of a video across visual frames, audio and transcripts, instead of sampling every video at a fixed one frame per second. The change is live now through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform.

What the numbers say

Google reports up to 88 percent fewer tokens, up to 66 percent lower analysis cost, and up to 7 percent higher accuracy against the previous static-sampling behaviour. Those are Google's own figures, and the qualifier "up to" is doing work: the gain depends on the video and the question. Google says the effect is most pronounced on long-form content, and lists 10-minute how-to guides, 90-minute lectures, and multi-hour recordings as the cases where the difference shows up.

Which models get it

The capability lands in Gemini 3.7 Flash, Gemini 3.6 Flash, and Gemini 3.5 Flash-Lite. Google labels the mode a Generative AI Preview and charges no extra fee for the feature itself on the API, so a developer already paying for these models pays only for the tokens the agent chooses to spend.

Google says the same capability will roll out in the Gemini app for general users soon, and will power the Ask YouTube feature on video watch pages in the coming months. No date is given for either.

What it means for a team building software

The old cost of using a video with a model was mostly a fixed cost of feeding every frame through the encoder, which made long videos disproportionately expensive to reason over. Under the new mode the model spends tokens on the parts it decides to look at, which is the shape of a browsing agent rather than a bulk transcriber.

That changes what is worth building. A support agent that answers a question from a two-hour product recording, a QA step that inspects a specific moment in a screen capture, or an analysis over a lecture backlog all become priceable when the token bill is set by the specific spans the model reads, not by the runtime of the file. The 7 percent accuracy figure is smaller than the cost number but points the same way: skipping the padding and paying attention to the relevant segments beats sampling everything at the same rate.

The trade for a builder is that the model now decides which segments to read. That decision is not free to inspect, and a task where a specific frame must be inspected regardless of what the model thinks is worth a look still needs to be pinned by prompt or by cutting the video before it goes in.

Source

• Introducing agentic video understanding with Gemini, Google, 1 September 2026
• Google To Bring New Video Understanding Tech To Ask YouTube, Search Engine Journal