LLM local
TokenSpeed
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
★ 2,5 mil estrelasMIT
Ver o guia →mlc-llm é um projeto de LLM local de código aberto de mlc-ai (mlc-ai/mlc-llm no GitHub). Tem 23.213 estrelas no GitHub e 2.148 forks, é publicado sob a licença Apache-2.0 e foi atualizado pela última vez no GitHub em 6 de outubro de 2026.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
Decide where the model must run - NVIDIA or AMD GPU, Apple Silicon, mobile, or WebGPU in the browser.
Clone the repository and follow the platform-specific installation guide linked from the README.
Install the compiled runtime for your platform and download or convert the model you want to serve.
Start MLCEngine with your model and send a chat request through the REST server or language API.
Compiles models for each target so they run natively instead of through a generic interpreter.
MLCEngine provides the same high-performance inference core across all supported platforms.
Serve models behind an OpenAI-style API, with Python, JavaScript, iOS and Android interfaces on the same engine.
Runs on NVIDIA (CUDA, Vulkan), AMD (ROCm, Vulkan), Apple (Metal) and Intel (Vulkan) GPUs depending on platform.
Deploy on iOS and iPadOS via Metal and on Android via OpenCL on Adreno and Mali GPUs.
The platform matrix includes web browsers via WebGPU and WASM, with the related WebLLM project for in-browser use.
Get the source code from GitHub and enter the project directory.
git clone https://github.com/mlc-ai/mlc-llm.git
cd mlc-llmThe README links to platform-specific install docs rather than one universal command, so follow the guide for your OS and GPU.
# Follow the installation guide for your platform - see the docs link in the READMEThe documentation's quick start walks through downloading a model and chatting with it via MLCEngine.
# Follow the quick start in the docs linked from the READMEIt is developed by the MLC AI community (the team behind TVM's machine learning compilation work) on GitHub.
Yes - MLC LLM is Apache-2.0 licensed and free to use, including commercially, under that license's terms.
No paid API key is required: MLC LLM runs open models locally on your own hardware. You only need model weights, which are downloaded from sources such as Hugging Face.
Compared with Ollama, which focuses on easy local serving, MLC LLM is a compiler stack that targets many more platforms - mobile and browsers included. Compared with vLLM, it trades a server-only focus for cross-platform reach.
Clone the repo, then follow the installation and quick start pages linked from the README for your platform, and run the documented first chat example.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
Lightweight inference engine for running LLMs efficiently on any device.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Distribute and run LLMs with a single file.
Discover and run local AI apps, serving optimized LLMs from your own GPU and NPU.
Let any LLM actually watch a video — frames plus transcript, run locally.
Os outros projetos de LLM local com mais estrelas no catálogo do LibreAI — a lista realista de alternativas se mlc-llm não for o ideal para você.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projetos de LLM local próximos no catálogo, os mais próximos em estrelas de mlc-llm: continue explorando dos dois lados dele.
An open-source RAG-based tool for chatting with your documents.
🔥 MaxKB is an open-source platform for building enterprise-grade agents. 强大易用的开源企业级智能体平台。
Distribute and run LLMs with a single file.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Open-source 24/7 Cowork app for OpenClaw, Hermes, Claude Code, Codex, OpenCode and 20+ more CLI Agent | Customize your assistants | Team them up｜Star if you …
High-performance In-browser LLM Inference Engine
Grátis e aberto: pegue o código-fonte, dê uma estrela no repositório e comece a criar.