LLM locaux
code-context-engine
Code-context-engine, in Python — built for save 94% on ai coding tokens. index your cod...
★ 425 étoilesMIT
Voir le guide →nano-vllm est un projet de LLM locaux open source, créé par GeeeekExplorer (GeeeekExplorer/nano-vllm sur GitHub). Il compte 15 743 étoiles sur GitHub et 2 666 forks, il est publié sous la licence MIT et a été mis à jour pour la dernière fois sur GitHub le 26 avril 2026.
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Choose nano-vllm to run offline inference with a codebase small enough to read fully, or to prototype engine ideas quickly.
Install nano-vllm directly from its GitHub repository with pip.
Fetch a model such as Qwen3-0.6B from Hugging Face to a local directory.
Load the model with the LLM class, set SamplingParams, and call generate on your prompts, as in example.py.
The whole implementation is about 1,200 lines of clean Python, so you can actually read how inference works.
The README benchmark shows output throughput comparable to vLLM on the tested setup (Qwen3-0.6B, RTX 4070 Laptop).
Reuses cached prefixes across requests to avoid recomputing shared prompt beginnings.
Splits a model across multiple GPUs with the tensor_parallel_size setting when one GPU is not enough.
Uses torch compilation and CUDA graphs as part of its optimization suite to cut inference overhead.
The LLM and SamplingParams interface mirrors vLLM, so existing vLLM-style code needs only minor changes.
Clone nano-vllm to read the source and run the included example and benchmark scripts.
git clone https://github.com/GeeeekExplorer/nano-vllm.git
cd nano-vllmInstall the package directly from the repository with pip.
pip install git+https://github.com/GeeeekExplorer/nano-vllm.gitDownload Qwen3-0.6B weights, then run example.py-style code with the LLM class and SamplingParams.
huggingface-cli download --resume-download Qwen/Qwen3-0.6B --local-dir ~/huggingface/Qwen3-0.6B/ --local-dir-use-symlinks FalseIt is a project by GeeeekExplorer with open-source contributors.
Yes. It is released under the MIT license.
No. It runs models locally on your GPU; you only download open model weights from Hugging Face.
Nano-vLLM trades vLLM's breadth (serving features, model coverage, production hardening) for a tiny, readable codebase with comparable offline throughput on the README's tested setup.
pip install it from GitHub, download Qwen3-0.6B, and run the example in example.py.
Code-context-engine, in Python — built for save 94% on ai coding tokens. index your cod...
Go-built AI gateway unifying many providers with routing and guardrails.
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
Early proposal and implementation for connecting LLMs to web pages.
Token efficient translation utility built on ChatGPT or any OpenAI compatible LLM.
Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
Les autres projets de la catégorie LLM locaux avec le plus d'étoiles du catalogue LibreAI — la sélection réaliste d'alternatives si nano-vllm ne vous convient pas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projets de la catégorie LLM locaux proches dans le catalogue, les plus proches en étoiles de nano-vllm — continuez à explorer de part et d'autre.
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
Access large language models from the command-line
High-performance In-browser LLM Inference Engine
Typescript/React Library for AI Chat 💬🚀
Gratuit et ouvert : récupérez le code source, mettez une étoile au dépôt et commencez à créer.