Local LLM
nobodywho
Lightweight inference engine for running LLMs efficiently on any device.
★ 1.5k starsEUPL-1.2
View guide →Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
mlc-llm is an open-source local llm project by mlc-ai (mlc-ai/mlc-llm on GitHub). It has 23,213 GitHub stars and 2,148 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 6 October 2026.
Decide where the model must run - NVIDIA or AMD GPU, Apple Silicon, mobile, or WebGPU in the browser.
Clone the repository and follow the platform-specific installation guide linked from the README.
Install the compiled runtime for your platform and download or convert the model you want to serve.
Start MLCEngine with your model and send a chat request through the REST server or language API.
Compiles models for each target so they run natively instead of through a generic interpreter.
MLCEngine provides the same high-performance inference core across all supported platforms.
Serve models behind an OpenAI-style API, with Python, JavaScript, iOS and Android interfaces on the same engine.
Runs on NVIDIA (CUDA, Vulkan), AMD (ROCm, Vulkan), Apple (Metal) and Intel (Vulkan) GPUs depending on platform.
Deploy on iOS and iPadOS via Metal and on Android via OpenCL on Adreno and Mali GPUs.
The platform matrix includes web browsers via WebGPU and WASM, with the related WebLLM project for in-browser use.
Get the source code from GitHub and enter the project directory.
git clone https://github.com/mlc-ai/mlc-llm.git
cd mlc-llmThe README links to platform-specific install docs rather than one universal command, so follow the guide for your OS and GPU.
# Follow the installation guide for your platform - see the docs link in the READMEThe documentation's quick start walks through downloading a model and chatting with it via MLCEngine.
# Follow the quick start in the docs linked from the READMEIt is developed by the MLC AI community (the team behind TVM's machine learning compilation work) on GitHub.
Yes - MLC LLM is Apache-2.0 licensed and free to use, including commercially, under that license's terms.
No paid API key is required: MLC LLM runs open models locally on your own hardware. You only need model weights, which are downloaded from sources such as Hugging Face.
Compared with Ollama, which focuses on easy local serving, MLC LLM is a compiler stack that targets many more platforms - mobile and browsers included. Compared with vLLM, it trades a server-only focus for cross-platform reach.
Clone the repo, then follow the installation and quick start pages linked from the README for your platform, and run the documented first chat example.
Lightweight inference engine for running LLMs efficiently on any device.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Distribute and run LLMs with a single file.
Discover and run local AI apps, serving optimized LLMs from your own GPU and NPU.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
Let any LLM actually watch a video — frames plus transcript, run locally.
The most-starred other Local LLM projects in the LibreAI catalogue — the realistic shortlist if mlc-llm is not the right fit.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Free and open — grab the source, star the repo, and start creating.