LibreAI / AI Agents / agent-memory-benchmark
// agents · vectorize-io/agent-memory-benchmark

agent-memory-benchmark

A benchmark for comparing AI agent memory systems head-to-head.

★ 144 stars⑂ 47 forksNOASSERTIONupdated 2026-10
▦ visual — this project's README ships no embeddable screenshot,
so here's a placeholder. The guide below is complete regardless.
// project facts

Project facts

agent-memory-benchmark is an open-source ai agents project by vectorize-io (vectorize-io/agent-memory-benchmark on GitHub). It has 144 GitHub stars and 47 forks, is released under the NOASSERTION license, and was last updated on GitHub on 7 October 2026.

GitHub stars
144
Forks
47
License
NOASSERTION
Last updated on GitHub
7 October 2026
Category
AI Agents
Listing
Full install guide
// how it works

How it works

  1. Define the job

    Know what you want the agent to accomplish before touching any code.

  2. Get the code

    Clone the repo and read the README's requirements section.

  3. Wire up dependencies

    Install what's needed and add API keys or configuration.

  4. Take it for a spin

    Run the quickstart example, then iterate from there.

// features

Why people love it

Standardized tasks

Same tests for every system.

Head-to-head comparisons

Apples-to-apples memory evals.

Memory-focused

Recall quality, not generic chat.

Open methodology

Inspect how scores are computed.

Vectorize-built

From the Hindsight team.

Active development

Updated as the field moves.

Advertisement
// run it

Run it yourself

  1. Step 1 — Clone the repo

    Download the source and move into the project folder.

    git clone https://github.com/vectorize-io/agent-memory-benchmark.git
    cd agent-memory-benchmark
  2. Step 2 — Create a venv and install

    Isolate dependencies, then install requirements.

    python -m venv .venv && source .venv/bin/activate
    pip install -r requirements.txt
  3. Step 3 — Run it

    Launch using the entry point shown in the README.

    python -m agent_memory_benchmark  # check README for the exact command
// use cases

What you can build

Advertisement
// faq

Questions, answered

Who maintains agent-memory-benchmark?

It is an open-source project at vectorize-io/agent-memory-benchmark with 47 forks. Check the contributors tab and recent commits to judge how active maintenance is.

Is agent-memory-benchmark free to use?

GitHub lists no license for this repo, so legally it's all rights reserved — check the README or ask the maintainer before building on it.

What does it measure?

How well agent memory systems recall and use information — standardized tasks for head-to-head comparison.

Who's it for?

Researchers and vendors who want memory performance as numbers, not claims.

How do I get started?

Clone the repo and follow the README quickstart — the README is the best starting point.

// keep exploring

Similar tools

Agents

trpc-agent-go

Go framework for production agent systems: graphs, tools, memory, A2A, MCP.

★ 1.8k starsApache-2.0
View guide →
AI Agents

Agent_Memory_Techniques

30 runnable notebooks teaching everything about agent memory.

★ 1.1k starsApache-2.0
View guide →
AI Agents

webarena-verified

WebArena-Verified: the audited, deterministically scored release of the WebArena web-agent benchmark.

★ 62 starsApache-2.0
View guide →
AI Agents

OpenViking

A self-evolving context database unifying agent memory, RAG, and skills.

★ 39.4k starsAGPL-3.0
View guide →
AI Agents

memanto

A memory agent for your agent fleet: it curates, reconciles, and expires shared memories across tools.

★ 2.3k starsMIT
View guide →
AI Agents

tau2-bench

A benchmark for tool-agent-user interaction in real-world domains.

★ 2.2k starsMIT
View guide →
// alternatives

Alternatives in AI Agents

The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if agent-memory-benchmark is not the right fit.

Get agent-memory-benchmark

Free and open — grab the source, star the repo, and start creating.