AI Agents
open-codex-computer-use
Open-source Computer Use as an MCP service: let any agent drive macOS, Linux, and Windows desktops.
★ 2.4k starsMIT
View guide →Reliable computer use for any agent: drive real apps through accessibility trees, not pixel guessing.
agent-desktop is an open-source ai agents project by lahfir (lahfir/agent-desktop on GitHub). It has 1,778 GitHub stars and 127 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 6 October 2026.
Decide which desktop workflow your agent must perform and which apps it involves.
Install agent-desktop globally from npm, or build it from source with cargo.
Grant macOS Accessibility permission (and Screen Recording if you need screenshots), using the built-in permission request command.
Take a snapshot of the target app, note element references, act on them, and re-snapshot to verify the result, optionally inside a traced session.
Snapshots return compact element references tied to a snapshot id, so agents act on identified controls rather than screen coordinates.
Dense apps can be explored through a shallow skeleton overview with targeted drill-down, which the project says cuts snapshot token use dramatically.
Reference-based actions use accessibility APIs and avoid silently stealing focus, cursor, keyboard, or clipboard.
Dozens of commands cover observation, interaction, keyboard, mouse, notifications, clipboard, window management, sessions, and trace export, all with structured JSON output.
Launching a Chromium app with CDP opens a verified DevTools port so Playwright, Puppeteer, or agent-browser can drive web content while native UI stays on the accessibility path.
Shared sessions give multi-agent workflows a common snapshot namespace and JSONL traces exportable as a single HTML viewer, and a C-ABI library supports in-process use from many languages.
The npm install is the recommended route; source builds need Rust 1.89+ and macOS 13+.
git clone https://github.com/lahfir/agent-desktop.git
cd agent-desktop
cargo build --releaseThe package downloads the prebuilt binary automatically.
npm install -g agent-desktop
agent-desktop permissions --requestList a Finder window's interactive elements, then click one by reference.
agent-desktop snapshot --app Finder -i
agent-desktop click @e3 --snapshot <snapshot_id>It is maintained by lahfir on GitHub.
Yes, it is Apache-2.0 licensed and free; the AI model that decides what to click comes from your own agent setup.
Agent Desktop needs no API key; it is a local control tool. The agent driving it uses your existing model access.
Screenshot-based computer use guesses coordinates from pixels and breaks when layouts shift; Agent Desktop reads the real accessibility tree and acts on stable element references that are safe to retry.
Run npm install -g agent-desktop, grant Accessibility permission, then take your first snapshot with agent-desktop snapshot --app Finder -i.
Open-source Computer Use as an MCP service: let any agent drive macOS, Linux, and Windows desktops.
ESP32 iPhone Agent — let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, no computer, no jailbreak
Benchmark computer-use agents on long, real-world desktop tasks.
AI agents that use real Android and iOS apps like a human — from minitap.
Pinvou Agent — open-source desktop AI agent for tools, files, knowledge, workflows and real deliverables
A desktop-grade AI agent from NetEase Youdao that does real work — analysis, slides, docs, video and research.
The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if agent-desktop is not the right fit.
A personal AI that really does things — on any OS, with your data staying yours.
An agentic skills framework & software development methodology that works.
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
The agent that grows with you — a Python framework for adaptive AI agents.
DeepSeek's plugin-based agent harness where everything is a plugin.
Free and open — grab the source, star the repo, and start creating.