LLM local
airllm
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
★ 35,5 mil estrelasApache-2.0
Ver o guia →ChatGLM2-6B é um projeto de LLM local de código aberto de zai-org (zai-org/ChatGLM2-6B no GitHub). Tem 15.501 estrelas no GitHub e 1.780 forks, é publicado sob a licença Apache-2.0 e foi atualizado pela última vez no GitHub em 27 de junho de 2024.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
ChatGLM2-6B suits bilingual chat on modest hardware; if you need the strongest current model, the README points to the GLM-4 open series instead.
Get the code, demos and requirements file from GitHub; the weights are downloaded from Hugging Face (THUDM/chatglm2-6b).
Install the Python requirements, plus FastAPI and Uvicorn if you want the API demos.
Start the CLI or web demo for a first conversation, or the API server to integrate the model into your own app.
Trained for Chinese and English dialogue, with a smooth chat experience and a low deployment threshold inherited from the first ChatGLM.
FlashAttention-based training extends the base context from 2K to 32K, and a separate ChatGLM2-6B-32K variant targets long-text understanding.
Multi-Query Attention gives faster inference and lower memory use than the first generation - the README reports a 42% speed-up in the official implementation.
Runs quantized in as little as about 6GB of GPU memory, with the supported conversation length at INT4 raised from 1K to 8K tokens.
The repo includes cli_demo.py, web_demo.py, an api.py FastAPI server and an OpenAI-compatible API demo.
Supports splitting the model across several smaller GPUs with Accelerate when no single card can hold it.
Clone the ChatGLM2-6B code and demo scripts.
git clone https://github.com/zai-org/ChatGLM2-6B.git
cd ChatGLM2-6BInstall the Python requirements from the repo.
pip install -r requirements.txtStart a terminal conversation with the model (weights download from Hugging Face on first load).
python cli_demo.pyThe repo is under zai-org, from the team behind the GLM model family (Zhipu AI and Tsinghua KEG). The README now directs users to the newer GLM-4 open models.
The code is Apache-2.0. The weights are free for academic research, and free for commercial use after completing the project's registration form, under its Model License.
No. You download the open weights and run them on your own hardware.
It was a leading 6B bilingual model of its generation, but newer models such as the GLM-4 series and current Qwen releases are stronger; ChatGLM2-6B remains attractive for its small footprint and bilingual tuning.
Clone the repo, run pip install -r requirements.txt, then python cli_demo.py or python web_demo.py for the browser UI.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
Os outros projetos de LLM local com mais estrelas no catálogo do LibreAI — a lista realista de alternativas se ChatGLM2-6B não for o ideal para você.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projetos de LLM local próximos no catálogo, os mais próximos em estrelas de ChatGLM2-6B: continue explorando dos dois lados dele.
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Access large language models from the command-line
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Typescript/React Library for AI Chat 💬🚀
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install
Grátis e aberto: pegue o código-fonte, dê uma estrela no repositório e comece a criar.