Инструменты разработчика
trafilatura
Trafilatura — a Python and CLI tool to crawl the web and extract clean text and metadata for corpora and RAG.
★ 6,9 тыс. звёздApache-2.0
Смотреть руководство →Scrapling — проект с открытым исходным кодом из категории «Инструменты разработчика» от D4Vinci (D4Vinci/Scrapling на GitHub). У него 86 348 звёзд на GitHub и 8 855 форков, он распространяется по лицензии BSD-3-Clause и последний раз обновлялся на GitHub 8 октября 2026.
Adaptive web scraping framework in Python — from a single request to a full-scale crawl
Decide what you want from Scrapling and check the README and requirements to confirm it fits your setup and Python environment.
Clone the repository from GitHub (github.com/D4Vinci/Scrapling) or use the documented install channel if you do not need the source.
Follow the README quickstart to install dependencies and set any required configuration or credentials.
Work through the README's first example end to end, then adapt it to your own project or data.
Selectors adapt when page structure changes, reducing broken scrapers.
Includes a Spiders framework for full-scale crawls, not just single-page fetches.
Advanced fetching with session support and stealth options built on Playwright.
Ships AI-oriented extras, including an MCP server and RAG-building support.
Async session management examples for high-throughput scraping.
Provides a CLI and an interactive shell for quick extraction jobs.
Start by getting the source and entering the project directory.
git clone https://github.com/D4Vinci/Scrapling.git
cd ScraplingInstall the core package, or the fetchers extra for browser-based scraping.
pip install scraplingRequired after installing the fetchers extra, per the README.
scrapling installIt is maintained by D4Vinci and contributors on GitHub, where the repository github.com/D4Vinci/Scrapling is the canonical home for code, issues and releases.
Yes — it is open source under the BSD-3-Clause license. Read the license in the repository for what is permitted, including any conditions on redistribution or commercial use.
No. Scrapling is a local library; optional AI extras use your own model access if you enable them.
Compared with BeautifulSoup or Scrapy: Scrapling adds adaptive selectors and stealth browser fetching on top of a Scrapy-like crawling model.
Clone github.com/D4Vinci/Scrapling, follow the README quickstart to install it, and run the documented first example before adapting it to your own work.
Trafilatura — a Python and CLI tool to crawl the web and extract clean text and metadata for corpora and RAG.
Scrape any website with a prompt: LLM-powered scraping pipelines in a few lines of Python
Skybridge: the full-stack TypeScript framework for building type-safe MCP Apps and ChatGPT Apps with React.
Build machine-learning web demos in Python within minutes — the fastest way to share a model with the world.
A Python metasearch library aggregating many web search services — plus CLI, API and MCP server
Local-first web search, fetch, crawl and research for AI coding agents over MCP — no API keys.
Другие проекты из категории «Инструменты разработчика» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если Scrapling вам не подходит.
The open source coding agent
The model-definition framework behind modern AI — thousands of pretrained transformers for text, vision, and audio.
Anthropic's agentic coding tool that lives in your terminal and works across your whole codebase
Cross-platform desktop all-in-one manager for Claude Code, Codex, OpenCode and other coding agents
OpenAI's lightweight coding agent that runs locally in your terminal
Соседние проекты из категории «Инструменты разработчика» в каталоге, ближайшие по числу звёзд к Scrapling — продолжайте изучать полку по обе стороны от него.
The agent development environment for running a fleet of parallel coding agents in worktrees
A free course to get into large language models, with roadmaps and runnable Colab notebooks
Production-grade engineering skills for AI coding agents, by Addy Osmani — 25 skills in one pack
CLI proxy that cuts LLM token consumption 60-90% on common dev commands — one Rust binary
An open-source AI agent that brings the power of Gemini directly into your terminal
Unified, efficient fine-tuning of 100+ LLMs and VLMs with a CLI and Web UI — ACL 2024
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.