ローカルLLM
code-context-engine
Code-context-engine, in Python — built for save 94% on ai coding tokens. index your cod...
★ 425 スターMIT
ガイドを見る →nano-vllmは、GeeeekExplorerが公開しているオープンソースのローカルLLMプロジェクトです（GitHub: GeeeekExplorer/nano-vllm）。GitHubスターは15,743、フォークは2,666で、MITライセンスの下で公開されています。GitHubでの最終更新は2026年4月26日です。
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Choose nano-vllm to run offline inference with a codebase small enough to read fully, or to prototype engine ideas quickly.
Install nano-vllm directly from its GitHub repository with pip.
Fetch a model such as Qwen3-0.6B from Hugging Face to a local directory.
Load the model with the LLM class, set SamplingParams, and call generate on your prompts, as in example.py.
The whole implementation is about 1,200 lines of clean Python, so you can actually read how inference works.
The README benchmark shows output throughput comparable to vLLM on the tested setup (Qwen3-0.6B, RTX 4070 Laptop).
Reuses cached prefixes across requests to avoid recomputing shared prompt beginnings.
Splits a model across multiple GPUs with the tensor_parallel_size setting when one GPU is not enough.
Uses torch compilation and CUDA graphs as part of its optimization suite to cut inference overhead.
The LLM and SamplingParams interface mirrors vLLM, so existing vLLM-style code needs only minor changes.
Clone nano-vllm to read the source and run the included example and benchmark scripts.
git clone https://github.com/GeeeekExplorer/nano-vllm.git
cd nano-vllmInstall the package directly from the repository with pip.
pip install git+https://github.com/GeeeekExplorer/nano-vllm.gitDownload Qwen3-0.6B weights, then run example.py-style code with the LLM class and SamplingParams.
huggingface-cli download --resume-download Qwen/Qwen3-0.6B --local-dir ~/huggingface/Qwen3-0.6B/ --local-dir-use-symlinks FalseIt is a project by GeeeekExplorer with open-source contributors.
Yes. It is released under the MIT license.
No. It runs models locally on your GPU; you only download open model weights from Hugging Face.
Nano-vLLM trades vLLM's breadth (serving features, model coverage, production hardening) for a tiny, readable codebase with comparable offline throughput on the README's tested setup.
pip install it from GitHub, download Qwen3-0.6B, and run the example in example.py.
Code-context-engine, in Python — built for save 94% on ai coding tokens. index your cod...
Go-built AI gateway unifying many providers with routing and guardrails.
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
Early proposal and implementation for connecting LLMs to web pages.
Token efficient translation utility built on ChatGPT or any OpenAI compatible LLM.
Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
LibreAIカタログに掲載されたローカルLLMプロジェクトの中で最もスターの多いもの — nano-vllmが合わなかった場合に検討したい、現実的な候補リストです。
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
カタログの中でnano-vllmとスター数が近いローカルLLMプロジェクト — 前後の棚もそのまま見て回れます。
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
Access large language models from the command-line
High-performance In-browser LLM Inference Engine
Typescript/React Library for AI Chat 💬🚀