LLM local
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
★ 39,6k estrellasApache-2.0
Ver la guía →BitNet es un proyecto de LLM local de código abierto de microsoft (microsoft/BitNet en GitHub). Tiene 41.232 estrellas en GitHub y 3.837 bifurcaciones, se publica bajo la licencia MIT y se actualizó por última vez en GitHub el 27 de julio de 2026.
BitNet — Microsoft's official inference framework for 1-bit large language models, running fast on ordinary CPUs, MIT
Clone with submodules, create the recommended conda environment, and install the Python requirements.
Fetch the official BitNet b1.58 2B GGUF weights from Hugging Face into a local models directory.
Point setup_env.py at the downloaded model with the i2_s quantisation and run inference locally.
Ternarised weights (-1, 0, 1) shrink the model so far that capable LLMs run fast on ordinary CPUs instead of needing a GPU.
The reference implementation for BitNet models, with the technical report (arXiv 2502.11880) behind the 1-bit approach.
A GPU kernel path exists alongside the CPU-first design for machines that do have a graphics card.
The same project line now includes 1-bit embeddings and a 1-bit speech-recognition engine (VibeASR.cpp), not just chat models.
The README build route: recursive clone, conda environment, requirements.
git clone --recursive https://github.com/microsoft/BitNet.git
conda create -n bitnet-cpp python=3.10
conda activate bitnet-cpp
pip install -r requirements.txtThe official 2B model weights from Hugging Face.
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4TPrepare the model directory for inference with the i2_s quantisation.
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_sMicrosoft, as the official inference framework for its 1-bit LLM research.
Yes — MIT licensed, and the model weights are distributed on Hugging Face under their own terms.
Model weights are ternarised to just three values (-1, 0, 1) instead of 16-bit floats, which slashes memory and lets CPUs run the model fast.
No — CPU inference is the point of the framework, though a GPU kernel path exists for machines that have one.
Clone recursively, set up the conda environment, download the BitNet-b1.58-2B-4T GGUF weights, and run setup_env.py against them.
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Access large language models from the command-line
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
vLLM-Omni — efficient inference for omni-modality models: image, audio, video, diffusion and policies.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Los demás proyectos de LLM local con más estrellas del catálogo de LibreAI: la lista realista de alternativas si BitNet no es lo que buscas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Proyectos de LLM local cercanos en el catálogo, los más próximos en estrellas a BitNet: sigue explorando a ambos lados.
Fabric is an open-source framework for augmenting humans using AI. It provides a modular system for solving specific problems using a crowdsourced set of AI …
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
Hundreds of models & providers. One command to find what runs on your hardware.
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Gratis y abierto: consigue el código fuente, dale una estrella al repositorio y empieza a crear.