Dev Tools
Scrapling
Adaptive web scraping framework in Python — from a single request to a full-scale crawl
★ 86.3k starsBSD-3-Clause
View guide →Scrape any website with a prompt: LLM-powered scraping pipelines in a few lines of Python
Scrapegraph-ai is an open-source dev tools project by ScrapeGraphAI (ScrapeGraphAI/Scrapegraph-ai on GitHub). It has 31,623 GitHub stars and 3,221 forks, is released under the MIT license, and was last updated on GitHub on 6 October 2026.
Decide what you want to achieve with Scrapegraph-ai and check the README on GitHub for requirements before installing.
Clone the repository from GitHub (or take the packaged install route in step 3 if the README offers one).
Follow the project's documented install commands — the Run section below lists the exact ones — and add any required API keys or models.
Work through the README's quickstart example end to end, then adapt it to your own project or data.
Describe the data you want in plain language; the LLM builds the extraction for you.
The flagship graph scrapes a page and returns structured output from a single prompt and URL.
Additional pipelines cover searching, multi-page scraping and other extraction patterns documented in the docs.
Works with different LLM providers and models through a simple config object.
Extracted, structured content drops neatly into RAG and data pipelines.
A hosted ScrapeGraph API and SDKs exist if you outgrow local runs (the open-source library remains MIT).
Get the library source and examples.
git clone https://github.com/ScrapeGraphAI/Scrapegraph-ai.git
cd Scrapegraph-aiInstall ScrapeGraphAI from PyPI.
pip install scrapegraphaiCreate a SmartScraperGraph with your model config, a prompt and a URL, then call run() as in the README usage example.
# graph = SmartScraperGraph(prompt='Extract the product names and prices', source='https://example.com', config=graph_config)
# result = graph.run() # see README Usage for the full snippetScrapegraph-ai is maintained by the ScrapeGraphAI organization/account on GitHub, together with open-source contributors, in the public repository ScrapeGraphAI/Scrapegraph-ai.
Yes — the code is open source under the MIT license. You can use, modify and redistribute it under that license's terms.
The library is free, but extraction is LLM-driven — you need your own LLM API key (or a local model), and you pay that provider's token costs.
Classic scrapers are free per run but break when layouts change; ScrapeGraphAI trades a small LLM cost per page for prompts that survive redesigns.
Clone the repository from GitHub, follow the install commands in the Run section (taken from the project's README), and complete the documented quickstart example before adapting it to your own use case.
Adaptive web scraping framework in Python — from a single request to a full-scale crawl
Expose your FastAPI endpoints as Model Context Protocol tools - with authentication - in a few lines of code.
Generates agent-first Go CLIs (plus MCP servers) for any API or website — with local SQLite sync built in
Turn any website into a CLI — and let AI agents drive your already-logged-in browser
⚡️next-generation personal AI assistant powered by LLM, RAG and agent loops, supporting computer-use, browser-use and coding agent, demo: https://demo.openag…
A bridge between Streamable HTTP / SSE and stdio MCP transports — connect any client to any server
The most-starred other Dev Tools projects in the LibreAI catalogue — the realistic shortlist if Scrapegraph-ai is not the right fit.
The open source coding agent
The model-definition framework behind modern AI — thousands of pretrained transformers for text, vision, and audio.
Anthropic's agentic coding tool that lives in your terminal and works across your whole codebase
Cross-platform desktop all-in-one manager for Claude Code, Codex, OpenCode and other coding agents
OpenAI's lightweight coding agent that runs locally in your terminal
Free and open — grab the source, star the repo, and start creating.