開発ツール
ai-game-devtools
Resource hub for AI-powered game development — LLMs, world models, agents.
★ 1.4k スターMIT
ガイドを見る →evalscopeは、modelscopeが公開しているオープンソースの開発ツールプロジェクトです（GitHub: modelscope/evalscope）。GitHubスターは3,509、フォークは509で、Apache-2.0ライセンスの下で公開されています。GitHubでの最終更新は2026年10月8日です。
A streamlined framework for evaluating and benchmarking LLMs, VLMs and AIGC models.
Decide the first concrete outcome you want from it, so you can pick the right example, template or mode instead of exploring everything at once.
Clone the repository (or install the published package) and skim its README quick start for the requirements for your platform.
Run the documented install step and set any required configuration — such as provider keys, data sources or local services — before the first run.
Execute the smallest documented example end to end, confirm the output is what you expected, then adapt it to your real data or workflow.
Industry-recognised benchmarks including MMLU, C-Eval and GSM8K built in, with custom dataset support.
Evaluate LLMs, VLMs, embedding, reranker and AIGC models in one framework.
Native backend plus OpenCompass, VLMEvalKit and RAGEval integrations.
Drive benchmarks such as GSM8K, AIME and SWE-bench Agentic inside a controlled multi-turn AgentLoop with Docker sandbox and full traces.
Stress-test model services with TTFT, TPOT and throughput metrics, recordable to wandb/swanlab.
Interactive web dashboard for comparison, plus pairwise Arena mode to rank models head-to-head.
Get the source so you can inspect, configure and run it locally.
git clone https://github.com/modelscope/evalscope.git
cd evalscopeInstall the package from PyPI.
pip install evalscopeRun a first benchmark against an OpenAI-compatible endpoint.
evalscope eval --model your-model-name --api-url $OPENAI_API_BASE_URL --api-key $OPENAI_API_KEY --eval-type openai_api --datasets gsm8k --limit 5It is maintained by modelscope on GitHub (modelscope/evalscope), where the source, issues and releases live.
Yes — it is free and open source under Apache-2.0, which permits use, modification and commercial redistribution, with licence/notice retention.
For API models you supply that provider's key; local models downloaded from ModelScope need no paid key, just compute.
Compared with OpenCompass or lm-eval used standalone, evalscope's distinguishing focus is: a streamlined framework for evaluating and benchmarking LLMs, VLMs and AIGC models. Pick it when that focus matches your need; the alternative may suit you better if its broader or different scope is the priority.
Clone https://github.com/modelscope/evalscope (or install its published package), follow the README quick start to install and configure it, then run the smallest documented example — the Run section on this page mirrors those steps.
Resource hub for AI-powered game development — LLMs, world models, agents.
Fine-tune and deploy 600+ LLMs and 300+ multimodal models with PEFT or full-parameter training.
Unified, efficient fine-tuning of 100+ LLMs and VLMs with a CLI and Web UI — ACL 2024
A coding-agent harness optimized for smaller local LLMs — small models, serious coding work
Backend.AI is a streamlined, container-based computing cluster platform.
An end-to-end framework for building custom AI applications and agents on top of your database.
LibreAIカタログに掲載された開発ツールプロジェクトの中で最もスターの多いもの — evalscopeが合わなかった場合に検討したい、現実的な候補リストです。
The open source coding agent
The model-definition framework behind modern AI — thousands of pretrained transformers for text, vision, and audio.
Anthropic's agentic coding tool that lives in your terminal and works across your whole codebase
Cross-platform desktop all-in-one manager for Claude Code, Codex, OpenCode and other coding agents
OpenAI's lightweight coding agent that runs locally in your terminal
カタログの中でevalscopeとスター数が近い開発ツールプロジェクト — 前後の棚もそのまま見て回れます。
Open-source AI coding assistant for VS Code. Code completion, chat, edits and reviews with local or hosted models. Your models, your infrastructure.
A Node.js crawler that turns websites into LLM-ready data and extracts structured SERP results.
A virtual terminal and filesystem that lets AI agents work across S3, Drive, Slack, Gmail and more.
Enterprise-grade codebase intelligence for AI agents — hybrid semantic search over 40M+ LOC, locally.
Like htop, but for AI coding agents — watch Claude Code and Codex sessions in real time.
A real-time architectural sensor, in pure Rust, that helps AI agents keep code quality from slipping.