LLM locaux
airllm
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
★ 35,5 k étoilesApache-2.0
Voir le guide →ChatGLM2-6B est un projet de LLM locaux open source, créé par zai-org (zai-org/ChatGLM2-6B sur GitHub). Il compte 15 501 étoiles sur GitHub et 1 780 forks, il est publié sous la licence Apache-2.0 et a été mis à jour pour la dernière fois sur GitHub le 27 juin 2024.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
ChatGLM2-6B suits bilingual chat on modest hardware; if you need the strongest current model, the README points to the GLM-4 open series instead.
Get the code, demos and requirements file from GitHub; the weights are downloaded from Hugging Face (THUDM/chatglm2-6b).
Install the Python requirements, plus FastAPI and Uvicorn if you want the API demos.
Start the CLI or web demo for a first conversation, or the API server to integrate the model into your own app.
Trained for Chinese and English dialogue, with a smooth chat experience and a low deployment threshold inherited from the first ChatGLM.
FlashAttention-based training extends the base context from 2K to 32K, and a separate ChatGLM2-6B-32K variant targets long-text understanding.
Multi-Query Attention gives faster inference and lower memory use than the first generation - the README reports a 42% speed-up in the official implementation.
Runs quantized in as little as about 6GB of GPU memory, with the supported conversation length at INT4 raised from 1K to 8K tokens.
The repo includes cli_demo.py, web_demo.py, an api.py FastAPI server and an OpenAI-compatible API demo.
Supports splitting the model across several smaller GPUs with Accelerate when no single card can hold it.
Clone the ChatGLM2-6B code and demo scripts.
git clone https://github.com/zai-org/ChatGLM2-6B.git
cd ChatGLM2-6BInstall the Python requirements from the repo.
pip install -r requirements.txtStart a terminal conversation with the model (weights download from Hugging Face on first load).
python cli_demo.pyThe repo is under zai-org, from the team behind the GLM model family (Zhipu AI and Tsinghua KEG). The README now directs users to the newer GLM-4 open models.
The code is Apache-2.0. The weights are free for academic research, and free for commercial use after completing the project's registration form, under its Model License.
No. You download the open weights and run them on your own hardware.
It was a leading 6B bilingual model of its generation, but newer models such as the GLM-4 series and current Qwen releases are stronger; ChatGLM2-6B remains attractive for its small footprint and bilingual tuning.
Clone the repo, run pip install -r requirements.txt, then python cli_demo.py or python web_demo.py for the browser UI.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
Les autres projets de la catégorie LLM locaux avec le plus d'étoiles du catalogue LibreAI — la sélection réaliste d'alternatives si ChatGLM2-6B ne vous convient pas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projets de la catégorie LLM locaux proches dans le catalogue, les plus proches en étoiles de ChatGLM2-6B — continuez à explorer de part et d'autre.
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Access large language models from the command-line
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Typescript/React Library for AI Chat 💬🚀
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install
Gratuit et ouvert : récupérez le code source, mettez une étoile au dépôt et commencez à créer.