LLM local
TokenSpeed
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
★ 2,5k estrellasMIT
Ver la guía →mlc-llm es un proyecto de LLM local de código abierto de mlc-ai (mlc-ai/mlc-llm en GitHub). Tiene 23.213 estrellas en GitHub y 2.148 bifurcaciones, se publica bajo la licencia Apache-2.0 y se actualizó por última vez en GitHub el 6 de octubre de 2026.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
Decide where the model must run - NVIDIA or AMD GPU, Apple Silicon, mobile, or WebGPU in the browser.
Clone the repository and follow the platform-specific installation guide linked from the README.
Install the compiled runtime for your platform and download or convert the model you want to serve.
Start MLCEngine with your model and send a chat request through the REST server or language API.
Compiles models for each target so they run natively instead of through a generic interpreter.
MLCEngine provides the same high-performance inference core across all supported platforms.
Serve models behind an OpenAI-style API, with Python, JavaScript, iOS and Android interfaces on the same engine.
Runs on NVIDIA (CUDA, Vulkan), AMD (ROCm, Vulkan), Apple (Metal) and Intel (Vulkan) GPUs depending on platform.
Deploy on iOS and iPadOS via Metal and on Android via OpenCL on Adreno and Mali GPUs.
The platform matrix includes web browsers via WebGPU and WASM, with the related WebLLM project for in-browser use.
Get the source code from GitHub and enter the project directory.
git clone https://github.com/mlc-ai/mlc-llm.git
cd mlc-llmThe README links to platform-specific install docs rather than one universal command, so follow the guide for your OS and GPU.
# Follow the installation guide for your platform - see the docs link in the READMEThe documentation's quick start walks through downloading a model and chatting with it via MLCEngine.
# Follow the quick start in the docs linked from the READMEIt is developed by the MLC AI community (the team behind TVM's machine learning compilation work) on GitHub.
Yes - MLC LLM is Apache-2.0 licensed and free to use, including commercially, under that license's terms.
No paid API key is required: MLC LLM runs open models locally on your own hardware. You only need model weights, which are downloaded from sources such as Hugging Face.
Compared with Ollama, which focuses on easy local serving, MLC LLM is a compiler stack that targets many more platforms - mobile and browsers included. Compared with vLLM, it trades a server-only focus for cross-platform reach.
Clone the repo, then follow the installation and quick start pages linked from the README for your platform, and run the documented first chat example.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
Lightweight inference engine for running LLMs efficiently on any device.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Distribute and run LLMs with a single file.
Discover and run local AI apps, serving optimized LLMs from your own GPU and NPU.
Let any LLM actually watch a video — frames plus transcript, run locally.
Los demás proyectos de LLM local con más estrellas del catálogo de LibreAI: la lista realista de alternativas si mlc-llm no es lo que buscas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Proyectos de LLM local cercanos en el catálogo, los más próximos en estrellas a mlc-llm: sigue explorando a ambos lados.
An open-source RAG-based tool for chatting with your documents.
🔥 MaxKB is an open-source platform for building enterprise-grade agents. 强大易用的开源企业级智能体平台。
Distribute and run LLMs with a single file.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Open-source 24/7 Cowork app for OpenClaw, Hermes, Claude Code, Codex, OpenCode and 20+ more CLI Agent | Customize your assistants | Team them up｜Star if you …
High-performance In-browser LLM Inference Engine
Gratis y abierto: consigue el código fuente, dale una estrella al repositorio y empieza a crear.