Lokale LLMs
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
★ 39,6 Tsd. SterneApache-2.0
Anleitung ansehen →BitNet ist ein Open-Source-Projekt aus der Kategorie Lokale LLMs von microsoft (microsoft/BitNet auf GitHub). Es hat 41.232 Sterne auf GitHub und 3.837 Forks, wird unter der Lizenz MIT veröffentlicht und wurde zuletzt am 27. Juli 2026 auf GitHub aktualisiert.
BitNet — Microsoft's official inference framework for 1-bit large language models, running fast on ordinary CPUs, MIT
Clone with submodules, create the recommended conda environment, and install the Python requirements.
Fetch the official BitNet b1.58 2B GGUF weights from Hugging Face into a local models directory.
Point setup_env.py at the downloaded model with the i2_s quantisation and run inference locally.
Ternarised weights (-1, 0, 1) shrink the model so far that capable LLMs run fast on ordinary CPUs instead of needing a GPU.
The reference implementation for BitNet models, with the technical report (arXiv 2502.11880) behind the 1-bit approach.
A GPU kernel path exists alongside the CPU-first design for machines that do have a graphics card.
The same project line now includes 1-bit embeddings and a 1-bit speech-recognition engine (VibeASR.cpp), not just chat models.
The README build route: recursive clone, conda environment, requirements.
git clone --recursive https://github.com/microsoft/BitNet.git
conda create -n bitnet-cpp python=3.10
conda activate bitnet-cpp
pip install -r requirements.txtThe official 2B model weights from Hugging Face.
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4TPrepare the model directory for inference with the i2_s quantisation.
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_sMicrosoft, as the official inference framework for its 1-bit LLM research.
Yes — MIT licensed, and the model weights are distributed on Hugging Face under their own terms.
Model weights are ternarised to just three values (-1, 0, 1) instead of 16-bit floats, which slashes memory and lets CPUs run the model fast.
No — CPU inference is the point of the framework, though a GPU kernel path exists for machines that have one.
Clone recursively, set up the conda environment, download the BitNet-b1.58-2B-4T GGUF weights, and run setup_env.py against them.
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Access large language models from the command-line
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Die anderen Projekte der Kategorie Lokale LLMs mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls BitNet nicht das Richtige ist.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Verwandte Projekte der Kategorie Lokale LLMs im Katalog, die BitNet bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
Fabric is an open-source framework for augmenting humans using AI. It provides a modular system for solving specific problems using a crowdsourced set of AI …
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
Hundreds of models & providers. One command to find what runs on your hardware.
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.