Локальные LLM
Web LLM
High-performance In-browser LLM Inference Engine
★ 19,2 тыс. звёздApache-2.0
Смотреть руководство →TokenSpeed — проект с открытым исходным кодом из категории «Локальные LLM» от lightseekorg (lightseekorg/tokenspeed на GitHub). У него 2 473 звезды на GitHub и 336 форков, он распространяется по лицензии MIT и последний раз обновлялся на GitHub 15 октября 2026.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
TokenSpeed’s docs cover installation and requirements for your GPU stack.
Follow the launching guide to serve a model with the engine.
Use the server parameters, model recipes and parallelism guides to fit long, tool-heavy agentic sessions.
Scheduling, KV cache ownership and overlap timing are modelled as a finite-state machine tuned for agent request patterns.
A pluggable kernel registry includes one of the fastest MLA implementations on NVIDIA Blackwell.
KV resource reuse safety is enforced by the type system at compile time, not by convention.
New agentic-coding models (Qwen3.8 Flash Next, GLM 5.3 Flash) were supported at release.
Start from the source and the linked guides.
git clone https://github.com/lightseekorg/tokenspeed.gitInstallation and first run, per the official docs.
https://lightseek.org/tokenspeed/guides/getting-startedThe launching guide covers serving configuration.
https://lightseek.org/tokenspeed/guides/launchingThe TokenSpeed Team at LightSeek (lightseekorg).
Yes, MIT-licensed and open source.
No — it is a self-hosted inference engine; you bring the hardware and the model weights.
vLLM is the general-purpose usability benchmark and TensorRT-LLM the performance one. TokenSpeed targets both at once, with an architecture specialised for agentic workloads rather than chat traffic.
Clone the repo and follow the official getting-started and launching guides at lightseek.org/tokenspeed.
High-performance In-browser LLM Inference Engine
A high-throughput and memory-efficient inference and serving engine for LLMs
Offline inference engine for art.
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots
Lightweight inference engine for running LLMs efficiently on any device.
Другие проекты из категории «Локальные LLM» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если TokenSpeed вам не подходит.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Соседние проекты из категории «Локальные LLM» в каталоге, ближайшие по числу звёзд к TokenSpeed — продолжайте изучать полку по обе стороны от него.
动手学Ollama，CPU玩转大模型部署，在线阅读地址：https://datawhalechina.github.io/handy-ollama/
LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llm…
Local-first widget tracking token usage and costs across 43+ AI coding tools.
Fast Rust scraper that turns web pages into clean data for LLMs.
AI e-book reader with semantic search, RAG chat, notes, and text-to-speech.
Noise-robust subtitle generator powered by Whisper and local ASR models.
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.