barehands turns a webcam into a hand-tracked display for AI assistants

barehands is an AGPL-3.0 project from Jared Rhodenizer that turns a laptop's webcam and a Chrome tab into a hand-tracked display that any AI assistant can control. The repository was created on 15 August 2026 and reached 898 stars and 197 forks by 6 September, a fork ratio of one in five that suggests real use rather than people adding a star and leaving.

Notes, images and 3D models appear over the camera feed as see-through cards. A pinch drag moves them, two hands scale, a flick throws them across the screen, and holding a claw shape for two seconds pulls a distant card into the user's hand. Palms together and fingers up clears the screen. The hand-shape gates are computed as ratios rather than pixel sizes, so they work at any camera distance.

What runs, and how it connects to an agent

The server is Python stdlib on port 8794, and the page loads Google MediaPipe for hand tracking and three.js for rendering from public CDNs, both permissively licensed. No headset, no controllers, no gloves.

The AI side is two small file-based protocols and no SDK. Writing thinking, idle, listening or speaking into a state file drives the on-screen ring, which represents the agent on the screen. A shell entry point accepts JSON commands, so a present verb moves a card to the centre of the screen, dims the rest, and highlights it; add_card, add_img, hand, explode, yank and hover place the other items. A read-back script prints what is currently on the screen so the model can check it before it responds. The server enforces an action allowlist and confines all media to a single folder.

Anything that can write a file or curl localhost can act as the controller: Claude Code, a local model, a cron job, a Stream Deck button. In a Claude Code session, two hooks copy the real agent state onto the on-screen ring.

Existing notes stay where they are

The notes orb reads a folder of Markdown, which means an existing Obsidian vault needs one config line and no import. Paths and orb kinds are declared in a small JSON file, one row per orb. The gestures were tuned on a real hand across weeks of use, and a troubleshooting document includes a debug overlay and pose sampler for cases where a gesture check fails on a different hand or a different camera.

The one real obstacle is the licence. AGPL-3.0-or-later means a business can run and modify it freely, but a modified version offered as a network service must open its source under the same terms. Rhodenizer mentions a paid closed-source licence path for embedding it inside a commercial product.

Source

jaredrhod/barehands on GitHub. Star, fork and creation dates from the GitHub REST API, verified 7 September 2026.