Lokale LLMs
airllm
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
★ 35,5 Tsd. SterneApache-2.0
Anleitung ansehen →ChatGLM2-6B ist ein Open-Source-Projekt aus der Kategorie Lokale LLMs von zai-org (zai-org/ChatGLM2-6B auf GitHub). Es hat 15.501 Sterne auf GitHub und 1.780 Forks, wird unter der Lizenz Apache-2.0 veröffentlicht und wurde zuletzt am 27. Juni 2024 auf GitHub aktualisiert.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
ChatGLM2-6B suits bilingual chat on modest hardware; if you need the strongest current model, the README points to the GLM-4 open series instead.
Get the code, demos and requirements file from GitHub; the weights are downloaded from Hugging Face (THUDM/chatglm2-6b).
Install the Python requirements, plus FastAPI and Uvicorn if you want the API demos.
Start the CLI or web demo for a first conversation, or the API server to integrate the model into your own app.
Trained for Chinese and English dialogue, with a smooth chat experience and a low deployment threshold inherited from the first ChatGLM.
FlashAttention-based training extends the base context from 2K to 32K, and a separate ChatGLM2-6B-32K variant targets long-text understanding.
Multi-Query Attention gives faster inference and lower memory use than the first generation - the README reports a 42% speed-up in the official implementation.
Runs quantized in as little as about 6GB of GPU memory, with the supported conversation length at INT4 raised from 1K to 8K tokens.
The repo includes cli_demo.py, web_demo.py, an api.py FastAPI server and an OpenAI-compatible API demo.
Supports splitting the model across several smaller GPUs with Accelerate when no single card can hold it.
Clone the ChatGLM2-6B code and demo scripts.
git clone https://github.com/zai-org/ChatGLM2-6B.git
cd ChatGLM2-6BInstall the Python requirements from the repo.
pip install -r requirements.txtStart a terminal conversation with the model (weights download from Hugging Face on first load).
python cli_demo.pyThe repo is under zai-org, from the team behind the GLM model family (Zhipu AI and Tsinghua KEG). The README now directs users to the newer GLM-4 open models.
The code is Apache-2.0. The weights are free for academic research, and free for commercial use after completing the project's registration form, under its Model License.
No. You download the open weights and run them on your own hardware.
It was a leading 6B bilingual model of its generation, but newer models such as the GLM-4 series and current Qwen releases are stronger; ChatGLM2-6B remains attractive for its small footprint and bilingual tuning.
Clone the repo, run pip install -r requirements.txt, then python cli_demo.py or python web_demo.py for the browser UI.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
Die anderen Projekte der Kategorie Lokale LLMs mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls ChatGLM2-6B nicht das Richtige ist.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Verwandte Projekte der Kategorie Lokale LLMs im Katalog, die ChatGLM2-6B bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Access large language models from the command-line
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Typescript/React Library for AI Chat 💬🚀
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.