ローカルLLM
airllm
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
★ 35.5k スターApache-2.0
ガイドを見る →ChatGLM2-6Bは、zai-orgが公開しているオープンソースのローカルLLMプロジェクトです（GitHub: zai-org/ChatGLM2-6B）。GitHubスターは15,501、フォークは1,780で、Apache-2.0ライセンスの下で公開されています。GitHubでの最終更新は2024年6月27日です。
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
ChatGLM2-6B suits bilingual chat on modest hardware; if you need the strongest current model, the README points to the GLM-4 open series instead.
Get the code, demos and requirements file from GitHub; the weights are downloaded from Hugging Face (THUDM/chatglm2-6b).
Install the Python requirements, plus FastAPI and Uvicorn if you want the API demos.
Start the CLI or web demo for a first conversation, or the API server to integrate the model into your own app.
Trained for Chinese and English dialogue, with a smooth chat experience and a low deployment threshold inherited from the first ChatGLM.
FlashAttention-based training extends the base context from 2K to 32K, and a separate ChatGLM2-6B-32K variant targets long-text understanding.
Multi-Query Attention gives faster inference and lower memory use than the first generation - the README reports a 42% speed-up in the official implementation.
Runs quantized in as little as about 6GB of GPU memory, with the supported conversation length at INT4 raised from 1K to 8K tokens.
The repo includes cli_demo.py, web_demo.py, an api.py FastAPI server and an OpenAI-compatible API demo.
Supports splitting the model across several smaller GPUs with Accelerate when no single card can hold it.
Clone the ChatGLM2-6B code and demo scripts.
git clone https://github.com/zai-org/ChatGLM2-6B.git
cd ChatGLM2-6BInstall the Python requirements from the repo.
pip install -r requirements.txtStart a terminal conversation with the model (weights download from Hugging Face on first load).
python cli_demo.pyThe repo is under zai-org, from the team behind the GLM model family (Zhipu AI and Tsinghua KEG). The README now directs users to the newer GLM-4 open models.
The code is Apache-2.0. The weights are free for academic research, and free for commercial use after completing the project's registration form, under its Model License.
No. You download the open weights and run them on your own hardware.
It was a leading 6B bilingual model of its generation, but newer models such as the GLM-4 series and current Qwen releases are stronger; ChatGLM2-6B remains attractive for its small footprint and bilingual tuning.
Clone the repo, run pip install -r requirements.txt, then python cli_demo.py or python web_demo.py for the browser UI.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
LibreAIカタログに掲載されたローカルLLMプロジェクトの中で最もスターの多いもの — ChatGLM2-6Bが合わなかった場合に検討したい、現実的な候補リストです。
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
カタログの中でChatGLM2-6Bとスター数が近いローカルLLMプロジェクト — 前後の棚もそのまま見て回れます。
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Access large language models from the command-line
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Typescript/React Library for AI Chat 💬🚀
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install