Agents
Data-Analysis-Agent
Chat with your data: an LLM agent that builds charts and business insights.
★ 2.6k starsCustom
View guide →Supercharge LLM app evaluations with objective metrics, synthetic test data and feedback loops.
ragas is an open-source ai agents project by vibrantlabsai (vibrantlabsai/ragas on GitHub). It has 15,966 GitHub stars and 1,750 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 24 February 2026.
Choose the aspects you need to measure, such as answer accuracy, faithfulness to retrieved context, or relevance for your RAG pipeline.
Add the library to your Python project from PyPI, or install it directly from the GitHub source.
Bring examples from your application, or let ragas generate a production-aligned test dataset covering a range of scenarios.
Run pre-built or custom metrics over your app's outputs, read the scores and reasons, and use them to tune prompts, retrieval and models.
Evaluate LLM applications with both LLM-based and traditional metrics instead of subjective manual review.
Automatically create comprehensive test datasets, including production-aligned test set generation when you have no dataset ready.
Define your own evaluators, such as an Aspect Critique built with DiscreteMetric, to score any aspect of an output with a reason.
The ragas quickstart command scaffolds a complete evaluation project, starting with a rag_eval template for RAG systems.
Works with popular LLM frameworks like LangChain and major observability tools, so scores fit into existing pipelines.
Leverage production data to continually improve your LLM applications rather than evaluating once and stopping.
Clone the ragas source if you want to develop against it; most users install from PyPI instead.
git clone https://github.com/vibrantlabsai/ragas.git
cd ragasInstall the released package into your Python environment.
pip install ragasUse the quickstart command to create a ready-made RAG evaluation project, then add your data and run it.
ragas quickstart rag_evalRagas is developed by vibrantlabsai (the team behind Ragas) together with open-source contributors.
Yes. The library is Apache-2.0 licensed and free, including for commercial use.
Often, yes: many ragas metrics use an LLM as the judge, and the quickstart examples expect a provider key such as OPENAI_API_KEY. You pay your provider for those calls; traditional metrics do not need one.
Ragas focuses on open-source, code-first metrics and test-set generation you run in your own pipeline, while hosted platforms bundle evaluation with tracing and dashboards; many teams use ragas metrics inside their existing observability stack.
Run pip install ragas, then ragas quickstart rag_eval to scaffold a project, set your model provider key, and score your first outputs.
Chat with your data: an LLM agent that builds charts and business insights.
Test prompts, agents and RAGs — LLM evals, red teaming and model comparisons from the CLI
an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM
Microsoft's R&D-Agent: LLM agents that automate data-driven research, Kaggle modelling and LLM fine-tuning.
The open-source studio for labeling data of every type.
Programmable guardrails to keep LLM systems safe and on rails.
The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if ragas is not the right fit.
A personal AI that really does things — on any OS, with your data staying yours.
An agentic skills framework & software development methodology that works.
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
The agent that grows with you — a Python framework for adaptive AI agents.
DeepSeek's plugin-based agent harness where everything is a plugin.
Free and open — grab the source, star the repo, and start creating.