Google published Artemis to let coding agents test on real Android phones

Google published a new open-source project called Artemis on GitHub, an Apache-licensed Python framework that turns natural-language instructions into Android UI automation and exposes a Model Context Protocol server so coding agents like Claude Code, Codex, Antigravity and Windsurf can control a real Android device. The repository was created on 13 August, and had 417 stars, 42 forks and a push on 5 September when checked.

The repo is in the google organisation and carries the Apache-2.0 licence. It requires Python 3.12 or later, a connected Android device with USB debugging enabled or an emulator, and Google says the one-click startup script installs its own toolchain: ADB, scrcpy, FFmpeg and the uv Python installer.

What the tool does

Artemis takes a task written in plain English, operates the connected phone, and returns a structured report. Google's example prompt in the README is: "Build the latest changes into an APK, install it on the connected device, open the login screen with a test account, verify if there are any unexpected popups after login, and return screenshots of the final page." The agent then plans and executes the steps, keeps a session transcript with timestamps and screenshots, and can retrieve the crash stack from logcat when something breaks.

Two execution profiles are offered. Flash runs a single-model reactive loop at roughly 3 to 5 seconds per step, aimed at deterministic UI tasks. Pro is a multi-agent graph with a Planner, an Operator and a read-only Checker that verifies plan checkpoints, and Google says it handles workflows over 100 steps and supports long-running exploration.

The benchmark claim

Google states that Artemis achieved over 99% task completion on Google Research's AndroidWorld benchmark, a suite of over 100 multi-step tasks across more than 20 apps. That number comes from Google's own README and its own leaderboard chart, so treat it as a vendor benchmark until an independent run replicates it. The linked benchmark repository at google-research/android_world is real and publishes the task list, which makes the claim checkable rather than opaque.

The IDE integration

The Model Context Protocol server is the piece most likely to matter for anyone already using Claude Code or Codex. artemis mcp --install writes the server config for a chosen IDE and adds a rules.md file that tells the coding agent how to divide work between Flash and Pro, how to handle timing on transient UI, and when to fall back to coordinate-based locators. Supported IDEs listed in the README are Antigravity, Claude Code, Cursor, Codex, Windsurf, VS Code, Cline and Roo, and OpenClaw.

The roadmap lists an Android Studio plugin, iOS support and on-device vision models as next steps, none released yet. Neither is voice control.

Source

• google/artemis on GitHub
• AndroidWorld benchmark repository