KI-Agenten
agent-memory-benchmark
A benchmark for comparing AI agent memory systems head-to-head.
★ 144 SterneNOASSERTION
Anleitung ansehen →webarena-verified ist ein Open-Source-Projekt aus der Kategorie KI-Agenten von ServiceNow (ServiceNow/webarena-verified auf GitHub). Es hat 62 Sterne auf GitHub und 13 Forks, wird unter der Lizenz Apache-2.0 veröffentlicht und wurde zuletzt am 8. März 2026 auf GitHub aktualisiert.
WebArena-Verified: the audited, deterministically scored release of the WebArena web-agent benchmark.
Decide whether to evaluate on the full dataset or the smaller, harder subset for cheaper runs.
Install webarena-verified with pip, run it directly with uvx, or use the published Docker image.
Export the dataset, start the site environments you need with the env commands, and configure your agent's environment URLs and credentials.
Run your agent, capture its responses and network traces, and score tasks with eval-tasks or the Python API.
Every task, reference answer, and evaluator has been manually reviewed and corrected relative to the original release.
Evaluation replaces LLM-as-a-judge and substring matching with type-aware normalisation and structural comparison.
Agent runs can be scored without live web environments by replaying captured network traces.
A difficulty-prioritised subset of the full task set supports faster, lower-cost evaluation.
The CLI starts and stops the benchmark's websites, with optimised Docker images and an environment-control API and dashboard.
Evaluate from the command line, from Python through the WebArenaVerified API, or entirely inside the published container image.
Most users install the package or use uvx/Docker instead of cloning.
git clone https://github.com/ServiceNow/webarena-verified.git
cd webarena-verifiedOr skip installation entirely and call it through uvx.
pip install webarena-verifiedFetch the dataset, then score an example agent log as the README demonstrates.
webarena-verified dataset-get --output webarena-verified.json
uvx webarena-verified eval-tasks --task-ids 108 --output-dir examples/agent_logs/demoIt is maintained by ServiceNow, with academic guidance acknowledged in the README.
Yes, it is Apache-2.0 licensed and published on PyPI and as a Docker image; running the site environments and your agent uses your own infrastructure and model access.
No API key is required by the benchmark, and scoring is deterministic rather than LLM-judged; the agent you evaluate uses its own model credentials.
The original WebArena contained tasks and evaluators with known errors and relied partly on fuzzy matching; the Verified release audits every task and evaluator and scores deterministically, including offline from traces.
Run uvx webarena-verified --help, export the dataset with dataset-get, start a site with env start, and score a run with eval-tasks.
A benchmark for comparing AI agent memory systems head-to-head.
Recursive self-improving physical agent OS — verified, evidence-based execution for robots and simulators
A benchmark for tool-agent-user interaction in real-world domains.
WorkArena: ServiceNow's benchmark for how well web agents handle everyday knowledge-work tasks.
Alibaba's in-page GUI agent: control any web interface with natural language, from one script tag
Rakazo — an open-source, self-hosted AI agent for persistent teammates: bring your own model and computer, run it on web, desktop, iPhone and Android
Die anderen Projekte der Kategorie KI-Agenten mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls webarena-verified nicht das Richtige ist.
A personal AI that really does things — on any OS, with your data staying yours.
An agentic skills framework & software development methodology that works.
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
The agent that grows with you — a Python framework for adaptive AI agents.
DeepSeek's plugin-based agent harness where everything is a plugin.
Verwandte Projekte der Kategorie KI-Agenten im Katalog, die webarena-verified bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
Local-first AI memory that runs offline on just 8 GB of RAM.
Autonomous research loops, from topic to experiments.
A framework for agent nexuses where intelligence emerges from interaction.
The official Python SDK for building on Vapi's voice AI platform.
A personal AI assistant that puts security first.
Build, deploy, and monetize autonomous agents across Web3.
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.