LibreAI / AI Agents / AgentBench
// agents · THUDM/AgentBench

AgentBench

A rigorous benchmark for LLMs acting as agents.

★ 3.8k stars⑂ 288 forksApache-2.0updated 2026-02
▦ visual — this project's README ships no embeddable screenshot,
so here's a placeholder. The guide below is complete regardless.
// project facts

Project facts

AgentBench is an open-source ai agents project by THUDM (THUDM/AgentBench on GitHub). It has 3,765 GitHub stars and 288 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 8 February 2026.

GitHub stars
3,765
Forks
288
License
Apache-2.0
Last updated on GitHub
8 February 2026
Category
AI Agents
Listing
Full install guide
// how it works

How it works

  1. Start with a use case

    Pick one concrete task so you can judge whether the project fits.

  2. Fetch the source

    Clone the repo and check the prerequisites listed in the README.

  3. Set up the environment

    Install what it needs and configure credentials or model access.

  4. Try the sample workflow

    Run a minimal example end to end before wiring it into your stack.

// features

Why people love it

Agentic evaluation

Tests models as actors in environments.

ICLR 2024

Peer-reviewed benchmark design.

Multi-environment

Diverse tasks beyond static QA.

THUDM research

From a leading academic lab.

Python harness

Runnable evaluation code.

Apache-2.0

Permissive licensing.

Advertisement
// run it

Run it yourself

  1. Step 1 — Clone the repo

    Download the source and move into the project folder.

    git clone https://github.com/THUDM/AgentBench.git
    cd AgentBench
  2. Step 2 — Install

    Set up a virtual environment and install dependencies.

    python -m venv .venv && source .venv/bin/activate
    pip install -r requirements.txt
  3. Step 3 — Run tasks

    Evaluate a model on a task subset per the docs.

    # see README for the task runners
// use cases

What you can build

Advertisement
// faq

Questions, answered

Who maintains AgentBench?

It is maintained by THUDM at THUDM/AgentBench, with 288 forks.

Is AgentBench free to use?

Yes. It is released under the Apache-2.0 license, which permits commercial use, modification, and redistribution. Read the LICENSE file for the exact terms.

Do I need a paid API to use it?

You need model access to evaluate; the harness itself is free.

How does it compare to SWE-bench?

AgentBench covers diverse agentic environments; SWE-bench targets real-world software engineering specifically.

How do I get started?

Follow the repo docs to run a task subset against your model.

// keep exploring

Similar tools

AI Agents

WorkArena

WorkArena: ServiceNow's benchmark for how well web agents handle everyday knowledge-work tasks.

★ 275 starsNOASSERTION
View guide →
AI Agents

sglang

High-performance serving for the LLMs that power your agents.

★ 36.9k starsApache-2.0
View guide →
AI Agents

OSWorld

OSWorld — benchmark multimodal agents on open-ended tasks in real computer environments.

★ 3.2k starsApache-2.0
View guide →
Agents

deepteam

Open-source red-teaming framework that stress-tests LLMs and AI agents.

★ 3.0k starsApache-2.0
View guide →
Agents

AI-fundamentals

Chinese-language course on AI basics: GPU architecture, CUDA, LLMs, and agents.

★ 2.9k starsApache-2.0
View guide →
AI Agents

ADR

Uber's ADR: discover, observe, benchmark, and detect threats across enterprise AI agents, deployed at Uber.

★ 1.9k starsApache-2.0
View guide →
// alternatives

Alternatives in AI Agents

The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if AgentBench is not the right fit.

Get AgentBench

Free and open — grab the source, star the repo, and start creating.