開発ツール
trafilatura
Trafilatura — a Python and CLI tool to crawl the web and extract clean text and metadata for corpora and RAG.
★ 6.9k スターApache-2.0
ガイドを見る →Scraplingは、D4Vinciが公開しているオープンソースの開発ツールプロジェクトです（GitHub: D4Vinci/Scrapling）。GitHubスターは86,348、フォークは8,855で、BSD-3-Clauseライセンスの下で公開されています。GitHubでの最終更新は2026年10月8日です。
Adaptive web scraping framework in Python — from a single request to a full-scale crawl
Decide what you want from Scrapling and check the README and requirements to confirm it fits your setup and Python environment.
Clone the repository from GitHub (github.com/D4Vinci/Scrapling) or use the documented install channel if you do not need the source.
Follow the README quickstart to install dependencies and set any required configuration or credentials.
Work through the README's first example end to end, then adapt it to your own project or data.
Selectors adapt when page structure changes, reducing broken scrapers.
Includes a Spiders framework for full-scale crawls, not just single-page fetches.
Advanced fetching with session support and stealth options built on Playwright.
Ships AI-oriented extras, including an MCP server and RAG-building support.
Async session management examples for high-throughput scraping.
Provides a CLI and an interactive shell for quick extraction jobs.
Start by getting the source and entering the project directory.
git clone https://github.com/D4Vinci/Scrapling.git
cd ScraplingInstall the core package, or the fetchers extra for browser-based scraping.
pip install scraplingRequired after installing the fetchers extra, per the README.
scrapling installIt is maintained by D4Vinci and contributors on GitHub, where the repository github.com/D4Vinci/Scrapling is the canonical home for code, issues and releases.
Yes — it is open source under the BSD-3-Clause license. Read the license in the repository for what is permitted, including any conditions on redistribution or commercial use.
No. Scrapling is a local library; optional AI extras use your own model access if you enable them.
Compared with BeautifulSoup or Scrapy: Scrapling adds adaptive selectors and stealth browser fetching on top of a Scrapy-like crawling model.
Clone github.com/D4Vinci/Scrapling, follow the README quickstart to install it, and run the documented first example before adapting it to your own work.
Trafilatura — a Python and CLI tool to crawl the web and extract clean text and metadata for corpora and RAG.
Scrape any website with a prompt: LLM-powered scraping pipelines in a few lines of Python
Skybridge: the full-stack TypeScript framework for building type-safe MCP Apps and ChatGPT Apps with React.
Build machine-learning web demos in Python within minutes — the fastest way to share a model with the world.
A Python metasearch library aggregating many web search services — plus CLI, API and MCP server
Local-first web search, fetch, crawl and research for AI coding agents over MCP — no API keys.
LibreAIカタログに掲載された開発ツールプロジェクトの中で最もスターの多いもの — Scraplingが合わなかった場合に検討したい、現実的な候補リストです。
The open source coding agent
The model-definition framework behind modern AI — thousands of pretrained transformers for text, vision, and audio.
Anthropic's agentic coding tool that lives in your terminal and works across your whole codebase
Cross-platform desktop all-in-one manager for Claude Code, Codex, OpenCode and other coding agents
OpenAI's lightweight coding agent that runs locally in your terminal
カタログの中でScraplingとスター数が近い開発ツールプロジェクト — 前後の棚もそのまま見て回れます。
The agent development environment for running a fleet of parallel coding agents in worktrees
A free course to get into large language models, with roadmaps and runnable Colab notebooks
Production-grade engineering skills for AI coding agents, by Addy Osmani — 25 skills in one pack
CLI proxy that cuts LLM token consumption 60-90% on common dev commands — one Rust binary
An open-source AI agent that brings the power of Gemini directly into your terminal
Unified, efficient fine-tuning of 100+ LLMs and VLMs with a CLI and Web UI — ACL 2024