AI Agents
bisheng
Enterprise LLM app platform: GenAI workflows, RAG, agents, model management, evaluation, fine-tuning.
★ 12.0k starsApache-2.0
View guide →Test prompts, agents and RAGs — LLM evals, red teaming and model comparisons from the CLI
promptfoo is an open-source ai agents project by promptfoo (promptfoo/promptfoo on GitHub). It has 25,815 GitHub stars and 2,452 forks, is released under the MIT license, and was last updated on GitHub on 8 October 2026.
Decide what you want to achieve with promptfoo and check the README on GitHub for requirements before installing.
Clone the repository from GitHub (or take the packaged install route in step 3 if the README offers one).
Follow the project's documented install commands — the Run section below lists the exact ones — and add any required API keys or models.
Work through the README's quickstart example end to end, then adapt it to your own project or data.
Run the same test set across models and prompts to compare quality side by side.
Automated probes for prompt injection, jailbreaks and other AI security issues.
Evaluate retrieval pipelines and agent behaviors, not just single prompts.
Tests live in simple config files that version well in git.
Designed to run in pipelines so regressions block deploys.
Runs from your terminal with npx — no platform account required to start.
Get the Promptfoo source and examples.
git clone https://github.com/promptfoo/promptfoo.git
cd promptfooInstall globally from npm (brew and pip installs are also documented), or skip installation with npx.
npm install -g promptfooInitialize an example config, add your model key, and run the evaluation.
promptfoo init
promptfoo evalpromptfoo is maintained by the promptfoo organization/account on GitHub, together with open-source contributors, in the public repository promptfoo/promptfoo.
Yes — the code is open source under the MIT license. You can use, modify and redistribute it under that license's terms.
Promptfoo is free and MIT-licensed; evaluations call the models you test, so you need API keys for those providers (local/open models are also supported).
Those are hosted evaluation platforms; Promptfoo is a local-first, config-in-git CLI with strong red-teaming — lighter to adopt, and your tests live with your code.
Clone the repository from GitHub, follow the install commands in the Run section (taken from the project's README), and complete the documented quickstart example before adapting it to your own use case.
Enterprise LLM app platform: GenAI workflows, RAG, agents, model management, evaluation, fine-tuning.
A TypeScript toolkit for building agents: unified LLM API, loop, TUI, and CLI.
Open-source hub of integrations, SDK and CLI for building and deploying GPT/LLM chatbots and agents.
Open-source red-teaming framework that stress-tests LLMs and AI agents.
An MCP server and CLI that lets AI agents build, test and run iOS and macOS projects.
Test your agents against prompt-injection attacks.
The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if promptfoo is not the right fit.
A personal AI that really does things — on any OS, with your data staying yours.
An agentic skills framework & software development methodology that works.
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
The agent that grows with you — a Python framework for adaptive AI agents.
DeepSeek's plugin-based agent harness where everything is a plugin.
Free and open — grab the source, star the repo, and start creating.