AI Agents
crawl4ai
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
★ 84.7k starsApache-2.0
View guide →Turn any website into clean, agent-ready data with one API.
firecrawl is an open-source ai agents project by firecrawl (firecrawl/firecrawl on GitHub). It has 189,612 GitHub stars and 10,047 forks, is released under the AGPL-3.0 license, and was last updated on GitHub on 8 October 2026.
Know what you want the agent to accomplish before touching any code.
Clone the repo and read the README's requirements section.
Install what's needed and add API keys or configuration.
Run the quickstart example, then iterate from there.
Turns messy HTML into clean, LLM-friendly markdown.
Handles dynamic, JS-heavy pages — not just static HTML.
Search the web as part of the same pipeline.
Pull typed data out of pages, not just text.
Designed as infrastructure for AI agents.
Run it yourself under AGPL-3.0, or use the hosted cloud.
Download the source and move into the project folder.
git clone https://github.com/firecrawl/firecrawl.git
cd firecrawlPull in the Node packages the project needs.
npm installStart it with the project's documented command (often a dev server).
npm run dev # check README for the exact scriptIt is an open-source project at firecrawl/firecrawl with 10047 forks. Check the contributors tab and recent commits to judge how active maintenance is.
The code is AGPL-3.0: free to use and modify, but if you run a modified version as a network service you must share your source changes.
No — the AGPL-3.0 repo is self-hostable; firecrawl.dev is the managed option if you'd rather not run infrastructure.
Yes — rendering dynamic pages is one of its core features, unlike raw HTTP fetchers.
Clone the repo and follow the README quickstart — the docs at https://firecrawl.dev walk through setup.
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
Turn messy documents into clean, structured data for language models.
Production-ready agentic RAG behind a clean REST API.
Convert PDFs, DOCX, and HTML into clean, AI-ready documents.
Alibaba's in-page GUI agent: control any web interface with natural language, from one script tag
A portable AI agent runtime for the JVM — write one @Agent, run it on any Java AI stack.
The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if firecrawl is not the right fit.
A personal AI that really does things — on any OS, with your data staying yours.
An agentic skills framework & software development methodology that works.
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
The agent that grows with you — a Python framework for adaptive AI agents.
DeepSeek's plugin-based agent harness where everything is a plugin.
Free and open — grab the source, star the repo, and start creating.