browser-use released macos-harness, which lets an LLM control the whole Mac: Reveneau

Why it mattersTeams that already trust browser-use to control Chrome now have the same author's answer for the desktop, so there is less reason to build a custom Mac-control layer inside an agent.

The team behind browser-use, the widely-used framework for agent-driven web control, published a companion project called macos-harness. It is MIT-licensed and describes itself as the simplest, smallest harness that lets an LLM complete virtually any task on a Mac. The repository stands at 822 GitHub stars over its 30-day window.

Six basic commands

The whole tool is six commands: see to capture what is on screen, key and type for keyboard input, click for the mouse, ax to read the macOS accessibility tree, and script to run AppleScript or shell. Anything else the harness does, an agent builds from those. That is the deliberate design choice, and it is why the project can call itself small.

The read side relies on the accessibility tree rather than on vision alone, so a model does not have to look at a screenshot to know a button is labelled Save. The write side goes through the operating system's normal input paths, which means the harness can do only what the target application already allows.

What it runs on

The install path uses the uv package manager and expects Python 3.12 on a Mac. It is meant to be used together with a coding agent, and the README names Codex and Claude Code as the tested clients. There is a macos-harness doctor command that checks the accessibility permissions the harness needs before an agent tries to send input.

Telemetry is on by default. The README lists what it records: command category, success, duration, harness version, OS and architecture, and the client the agent is running under. It excludes prompts, app names and screenshots. Turning it off is a documented flag rather than a hidden setting.

Why it matters that this vendor released it

Desktop control by an agent is not a new idea. Anthropic offers Computer Use, various open projects build on pyautogui, and Apple has its own accessibility APIs. What is different here is that browser-use is a team teams already trust for web control, publishing the equivalent for the Mac under the same MIT terms and with the same minimal design. If you already have an agent using browser-use for the browser tab, macos-harness is the least-surprising choice for the rest of the machine.

If you were about to write your own macOS control layer to sit inside an agent, read this first. Six commands and a read path that uses accessibility data first are small enough to audit, and MIT means you can copy it into your own code if you need to. Any reason not to adopt it will be about policy: giving a language model direct keyboard and mouse control on a developer's Mac is a decision to make consciously, and the honest view is that this harness makes that easier rather than safer.

The doctor command and the toggle for telemetry are the two things worth adding to whatever internal process controls installing the package. Both are the sort of thing you want to have decided on before the first agent run, not after.

Source

Primary source: browser-use/macos-harness on GitHub.

This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.