Image: Vercel Labs
Why it mattersA team can let an agent drive a desktop app without screen pixels. The agent reads the app's own UI records. Actions can be retried without a new screenshot.
Vercel Labs released version 1.0 of Agent Desktop on 28 October 2026. The project is a Rust tool for AI agents. It lets an agent read and run a desktop app on macOS, Windows or Linux. The agent uses the operating system's accessibility tree, not a screen capture. The repo is at vercel-labs/agent-desktop on GitHub. The licence is Apache-2.0.
The GitHub page states what the tool does. The page says it gives any agent reliable computer use on the desktop. It also says references stay stable and actions stay safe to retry, rather than guessing from pixels.
How it works
The tool comes as a command-line program and a C library. The agent calls four commands in a loop. The project's docs name them. snapshot maps the UI. snapshot --root goes deeper into one part. click acts on a control. wait checks the result. The project calls this an observe-act-verify loop.
Vercel Labs lists three supported platforms on the docs site. macOS 13 and later on Intel or Apple Silicon, using the AX API. Windows 10 build 1809 and later on x64 or ARM64, using UI Automation. Ubuntu 24.04 and later on x86_64 or aarch64, using AT-SPI. The repo lists more than 50 commands. They cover snapshot, click, type, scroll, window and clipboard.
What is new in 1.0
Three releases went out in three days. GitHub lists v1.0.0 and v1.0.1 on 28 October 2026. v1.0.2 followed on 30 October 2026. The v1.0.2 notes point to the new documentation site at agent-browser.dev/desktop. The v1.0.1 notes say the team automated the publishing step. The repo was first created on 19 February 2026.
The repo had 1,945 stars and 134 forks when this item went to press. Trendshift's daily list ranked it third on the same day. The project name shares a root with Vercel Labs' agent-browser, a sister project for web pages.
The tool runs through npm as agent-desktop. The repo lists 12 open issues and reports 301 commits on the main branch. The topics on its GitHub page are accessibility, accessibility-api, ai-agents, computer-use, computer-use-agent, desktop-automation, macos, mcp and rust.
A team that gives agents work on real desktop apps can use the tree as its record of the UI. A pixel-based tool reads a screenshot and may click the wrong control after a design change. Agent Desktop names every button, field and menu. The agent reads from the same records a screen reader does. The project is new. The test is whether the records stay right on apps the team does not build.
Source
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.
