AI Agents
OSWorld-V2
Benchmark computer-use agents on long, real-world desktop tasks.
★ 353 starsApache-2.0
View guide →A benchmark testing whether agents can solve long-horizon coding tasks.
SWE-bench_Pro-os is an open-source ai agents project by scaleapi (scaleapi/SWE-bench_Pro-os on GitHub). It has 539 GitHub stars and 101 forks, is released under the MIT license, and was last updated on GitHub on 22 September 2026.
Decide what you want the project to do before installing anything.
Grab the source and skim the README for setup requirements.
Install dependencies and add any API keys or settings it needs.
Launch it and follow the example workflow from the docs.
Tasks beyond quick fixes.
Realistic software work.
Builds on a trusted benchmark.
From a leading data lab.
MIT licensed, inspectable.
Measures agents, not just models.
Download the source and move into the project folder.
git clone https://github.com/scaleapi/SWE-bench_Pro-os.git
cd SWE-bench_Pro-osCreate a virtual environment and install the requirements.
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txtTreat these as starting points - the README documents the exact commands.
python examples/quickstart.pyIt is an open-source project at scaleapi/SWE-bench_Pro-os with 101 forks. Check the repo's contributors and recent commits to judge how active maintenance is.
Yes - it is MIT licensed, so you can use, modify, and distribute it freely, including in commercial projects.
Running the evaluation is free; the model under test may use your own paid API keys.
Both test coding agents; SWE-bench Pro emphasizes long-horizon software engineering tasks in the SWE-bench tradition. Run both for a fuller picture.
Clone the repo, read the README's quickstart, and run the provided example - most projects get you to a first result in minutes.
Benchmark computer-use agents on long, real-world desktop tasks.
A control plane with a durable state kernel that keeps long-horizon agents moving across sessions.
The benchmark that grades AI on real GitHub issues.
An open benchmark for browser AI agents on everyday tasks.
WorkArena: ServiceNow's benchmark for how well web agents handle everyday knowledge-work tasks.
A self-improving agent for coding workflows and long-running autonomous tasks
The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if SWE-bench_Pro-os is not the right fit.
A personal AI that really does things — on any OS, with your data staying yours.
An agentic skills framework & software development methodology that works.
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
The agent that grows with you — a Python framework for adaptive AI agents.
DeepSeek's plugin-based agent harness where everything is a plugin.
Free and open — grab the source, star the repo, and start creating.