ИИ-агенты
SWE-bench_Pro-os
A benchmark testing whether agents can solve long-horizon coding tasks.
★ 539 звёздMIT
Смотреть руководство →MMLU-Pro — проект с открытым исходным кодом из категории «ИИ-агенты» от TIGER-AI-Lab (TIGER-AI-Lab/MMLU-Pro на GitHub). У него 429 звёзд на GitHub и 58 форков, он распространяется по лицензии Apache-2.0 и последний раз обновлялся на GitHub 18 марта 2026.
The tougher, more robust successor to the MMLU benchmark.
Write down the outcome you want in one sentence.
Clone it locally and check the README.
Get dependencies and tooling in place per the docs.
Follow the minimal example, then adapt it to your needs.
Beyond what MMLU could separate.
Ten options per question raise the bar.
Breadth across knowledge domains.
Code and data, open to all.
Peer-reviewed benchmark design.
Increasingly the test labs must pass.
Download the source and move into the project folder.
git clone https://github.com/TIGER-AI-Lab/MMLU-Pro.git
cd MMLU-ProIsolate dependencies, then install requirements.
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txtLaunch using the entry point shown in the README.
python -m mmlu_pro # check README for the exact commandIt is an open-source project at TIGER-AI-Lab/MMLU-Pro with 58 forks. Check the contributors tab and recent commits to judge how active maintenance is.
Yes — Apache 2.0 allows commercial use and modification. Keep the license and notice files, and note it includes a patent grant.
More answer choices per question and harder items overall — designed to separate frontier models.
Clone the repo and follow the eval instructions; it's code and data, not a service.
Clone the repo and follow the README quickstart — the README is the best starting point.
A benchmark testing whether agents can solve long-horizon coding tasks.
The benchmark that grades AI on real GitHub issues.
A token-conscious MCP server that connects AI tools to Postgres, MySQL, SQLite and more.
A benchmark for tool-agent-user interaction in real-world domains.
An open benchmark for browser AI agents on everyday tasks.
A graduate-level benchmark even Google can't crack.
Другие проекты из категории «ИИ-агенты» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если MMLU-Pro вам не подходит.
A personal AI that really does things — on any OS, with your data staying yours.
An agentic skills framework & software development methodology that works.
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
The agent that grows with you — a Python framework for adaptive AI agents.
DeepSeek's plugin-based agent harness where everything is a plugin.
Соседние проекты из категории «ИИ-агенты» в каталоге, ближайшие по числу звёзд к MMLU-Pro — продолжайте изучать полку по обе стороны от него.
A harness-oriented framework for production-grade agent applications.
Safety testing for agents, right in pytest.
Snowflake's Agent World Model: synthesise 1,000 executable MCP tool-use environments for agentic RL.
Agentic memory for LLM agents, from a published research project.
Guardrails for building secure, robust agents.
Benchmark computer-use agents on long, real-world desktop tasks.
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.