Локальные LLM
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
★ 39,6 тыс. звёздApache-2.0
Смотреть руководство →BitNet — проект с открытым исходным кодом из категории «Локальные LLM» от microsoft (microsoft/BitNet на GitHub). У него 41 232 звезды на GitHub и 3 837 форков, он распространяется по лицензии MIT и последний раз обновлялся на GitHub 27 июля 2026.
BitNet — Microsoft's official inference framework for 1-bit large language models, running fast on ordinary CPUs, MIT
Clone with submodules, create the recommended conda environment, and install the Python requirements.
Fetch the official BitNet b1.58 2B GGUF weights from Hugging Face into a local models directory.
Point setup_env.py at the downloaded model with the i2_s quantisation and run inference locally.
Ternarised weights (-1, 0, 1) shrink the model so far that capable LLMs run fast on ordinary CPUs instead of needing a GPU.
The reference implementation for BitNet models, with the technical report (arXiv 2502.11880) behind the 1-bit approach.
A GPU kernel path exists alongside the CPU-first design for machines that do have a graphics card.
The same project line now includes 1-bit embeddings and a 1-bit speech-recognition engine (VibeASR.cpp), not just chat models.
The README build route: recursive clone, conda environment, requirements.
git clone --recursive https://github.com/microsoft/BitNet.git
conda create -n bitnet-cpp python=3.10
conda activate bitnet-cpp
pip install -r requirements.txtThe official 2B model weights from Hugging Face.
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4TPrepare the model directory for inference with the i2_s quantisation.
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_sMicrosoft, as the official inference framework for its 1-bit LLM research.
Yes — MIT licensed, and the model weights are distributed on Hugging Face under their own terms.
Model weights are ternarised to just three values (-1, 0, 1) instead of 16-bit floats, which slashes memory and lets CPUs run the model fast.
No — CPU inference is the point of the framework, though a GPU kernel path exists for machines that have one.
Clone recursively, set up the conda environment, download the BitNet-b1.58-2B-4T GGUF weights, and run setup_env.py against them.
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Access large language models from the command-line
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Другие проекты из категории «Локальные LLM» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если BitNet вам не подходит.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Соседние проекты из категории «Локальные LLM» в каталоге, ближайшие по числу звёзд к BitNet — продолжайте изучать полку по обе стороны от него.
Fabric is an open-source framework for augmenting humans using AI. It provides a modular system for solving specific problems using a crowdsourced set of AI …
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
Hundreds of models & providers. One command to find what runs on your hardware.
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.