Local LLM
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
★ 39.6k starsApache-2.0
View guide →BitNet is an open-source local llm project by microsoft (microsoft/BitNet on GitHub). It has 41,232 GitHub stars and 3,837 forks, is released under the MIT license, and was last updated on GitHub on 27 July 2026.
BitNet — Microsoft's official inference framework for 1-bit large language models, running fast on ordinary CPUs, MIT
Clone with submodules, create the recommended conda environment, and install the Python requirements.
Fetch the official BitNet b1.58 2B GGUF weights from Hugging Face into a local models directory.
Point setup_env.py at the downloaded model with the i2_s quantisation and run inference locally.
Ternarised weights (-1, 0, 1) shrink the model so far that capable LLMs run fast on ordinary CPUs instead of needing a GPU.
The reference implementation for BitNet models, with the technical report (arXiv 2502.11880) behind the 1-bit approach.
A GPU kernel path exists alongside the CPU-first design for machines that do have a graphics card.
The same project line now includes 1-bit embeddings and a 1-bit speech-recognition engine (VibeASR.cpp), not just chat models.
The README build route: recursive clone, conda environment, requirements.
git clone --recursive https://github.com/microsoft/BitNet.git
conda create -n bitnet-cpp python=3.10
conda activate bitnet-cpp
pip install -r requirements.txtThe official 2B model weights from Hugging Face.
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4TPrepare the model directory for inference with the i2_s quantisation.
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_sMicrosoft, as the official inference framework for its 1-bit LLM research.
Yes — MIT licensed, and the model weights are distributed on Hugging Face under their own terms.
Model weights are ternarised to just three values (-1, 0, 1) instead of 16-bit floats, which slashes memory and lets CPUs run the model fast.
No — CPU inference is the point of the framework, though a GPU kernel path exists for machines that have one.
Clone recursively, set up the conda environment, download the BitNet-b1.58-2B-4T GGUF weights, and run setup_env.py against them.
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Access large language models from the command-line
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
The most-starred other Local LLM projects in the LibreAI catalogue — the realistic shortlist if BitNet is not the right fit.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Nearby Local LLM projects in the catalogue, closest in stars to BitNet — keep exploring the shelf either side of it.
Fabric is an open-source framework for augmenting humans using AI. It provides a modular system for solving specific problems using a crowdsourced set of AI …
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
Hundreds of models & providers. One command to find what runs on your hardware.
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Free and open — grab the source, star the repo, and start creating.