Локальные LLM
code-context-engine
Code-context-engine, in Python — built for save 94% on ai coding tokens. index your cod...
★ 425 звёздMIT
Смотреть руководство →nano-vllm — проект с открытым исходным кодом из категории «Локальные LLM» от GeeeekExplorer (GeeeekExplorer/nano-vllm на GitHub). У него 15 743 звезды на GitHub и 2 666 форков, он распространяется по лицензии MIT и последний раз обновлялся на GitHub 26 апреля 2026.
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Choose nano-vllm to run offline inference with a codebase small enough to read fully, or to prototype engine ideas quickly.
Install nano-vllm directly from its GitHub repository with pip.
Fetch a model such as Qwen3-0.6B from Hugging Face to a local directory.
Load the model with the LLM class, set SamplingParams, and call generate on your prompts, as in example.py.
The whole implementation is about 1,200 lines of clean Python, so you can actually read how inference works.
The README benchmark shows output throughput comparable to vLLM on the tested setup (Qwen3-0.6B, RTX 4070 Laptop).
Reuses cached prefixes across requests to avoid recomputing shared prompt beginnings.
Splits a model across multiple GPUs with the tensor_parallel_size setting when one GPU is not enough.
Uses torch compilation and CUDA graphs as part of its optimization suite to cut inference overhead.
The LLM and SamplingParams interface mirrors vLLM, so existing vLLM-style code needs only minor changes.
Clone nano-vllm to read the source and run the included example and benchmark scripts.
git clone https://github.com/GeeeekExplorer/nano-vllm.git
cd nano-vllmInstall the package directly from the repository with pip.
pip install git+https://github.com/GeeeekExplorer/nano-vllm.gitDownload Qwen3-0.6B weights, then run example.py-style code with the LLM class and SamplingParams.
huggingface-cli download --resume-download Qwen/Qwen3-0.6B --local-dir ~/huggingface/Qwen3-0.6B/ --local-dir-use-symlinks FalseIt is a project by GeeeekExplorer with open-source contributors.
Yes. It is released under the MIT license.
No. It runs models locally on your GPU; you only download open model weights from Hugging Face.
Nano-vLLM trades vLLM's breadth (serving features, model coverage, production hardening) for a tiny, readable codebase with comparable offline throughput on the README's tested setup.
pip install it from GitHub, download Qwen3-0.6B, and run the example in example.py.
Code-context-engine, in Python — built for save 94% on ai coding tokens. index your cod...
Go-built AI gateway unifying many providers with routing and guardrails.
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
Early proposal and implementation for connecting LLMs to web pages.
Token efficient translation utility built on ChatGPT or any OpenAI compatible LLM.
Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
Другие проекты из категории «Локальные LLM» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если nano-vllm вам не подходит.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Соседние проекты из категории «Локальные LLM» в каталоге, ближайшие по числу звёзд к nano-vllm — продолжайте изучать полку по обе стороны от него.
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
Access large language models from the command-line
High-performance In-browser LLM Inference Engine
Typescript/React Library for AI Chat 💬🚀
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.