LLM locaux
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
★ 39,6 k étoilesApache-2.0
Voir le guide →BitNet est un projet de LLM locaux open source, créé par microsoft (microsoft/BitNet sur GitHub). Il compte 41 232 étoiles sur GitHub et 3 837 forks, il est publié sous la licence MIT et a été mis à jour pour la dernière fois sur GitHub le 27 juillet 2026.
BitNet — Microsoft's official inference framework for 1-bit large language models, running fast on ordinary CPUs, MIT
Clone with submodules, create the recommended conda environment, and install the Python requirements.
Fetch the official BitNet b1.58 2B GGUF weights from Hugging Face into a local models directory.
Point setup_env.py at the downloaded model with the i2_s quantisation and run inference locally.
Ternarised weights (-1, 0, 1) shrink the model so far that capable LLMs run fast on ordinary CPUs instead of needing a GPU.
The reference implementation for BitNet models, with the technical report (arXiv 2502.11880) behind the 1-bit approach.
A GPU kernel path exists alongside the CPU-first design for machines that do have a graphics card.
The same project line now includes 1-bit embeddings and a 1-bit speech-recognition engine (VibeASR.cpp), not just chat models.
The README build route: recursive clone, conda environment, requirements.
git clone --recursive https://github.com/microsoft/BitNet.git
conda create -n bitnet-cpp python=3.10
conda activate bitnet-cpp
pip install -r requirements.txtThe official 2B model weights from Hugging Face.
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4TPrepare the model directory for inference with the i2_s quantisation.
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_sMicrosoft, as the official inference framework for its 1-bit LLM research.
Yes — MIT licensed, and the model weights are distributed on Hugging Face under their own terms.
Model weights are ternarised to just three values (-1, 0, 1) instead of 16-bit floats, which slashes memory and lets CPUs run the model fast.
No — CPU inference is the point of the framework, though a GPU kernel path exists for machines that have one.
Clone recursively, set up the conda environment, download the BitNet-b1.58-2B-4T GGUF weights, and run setup_env.py against them.
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Access large language models from the command-line
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Les autres projets de la catégorie LLM locaux avec le plus d'étoiles du catalogue LibreAI — la sélection réaliste d'alternatives si BitNet ne vous convient pas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projets de la catégorie LLM locaux proches dans le catalogue, les plus proches en étoiles de BitNet — continuez à explorer de part et d'autre.
Fabric is an open-source framework for augmenting humans using AI. It provides a modular system for solving specific problems using a crowdsourced set of AI …
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
Hundreds of models & providers. One command to find what runs on your hardware.
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Gratuit et ouvert : récupérez le code source, mettez une étoile au dépôt et commencez à créer.