Lokale LLMs
TokenSpeed
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
★ 2,5 Tsd. SterneMIT
Anleitung ansehen →mlc-llm ist ein Open-Source-Projekt aus der Kategorie Lokale LLMs von mlc-ai (mlc-ai/mlc-llm auf GitHub). Es hat 23.213 Sterne auf GitHub und 2.148 Forks, wird unter der Lizenz Apache-2.0 veröffentlicht und wurde zuletzt am 6. Oktober 2026 auf GitHub aktualisiert.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
Decide where the model must run - NVIDIA or AMD GPU, Apple Silicon, mobile, or WebGPU in the browser.
Clone the repository and follow the platform-specific installation guide linked from the README.
Install the compiled runtime for your platform and download or convert the model you want to serve.
Start MLCEngine with your model and send a chat request through the REST server or language API.
Compiles models for each target so they run natively instead of through a generic interpreter.
MLCEngine provides the same high-performance inference core across all supported platforms.
Serve models behind an OpenAI-style API, with Python, JavaScript, iOS and Android interfaces on the same engine.
Runs on NVIDIA (CUDA, Vulkan), AMD (ROCm, Vulkan), Apple (Metal) and Intel (Vulkan) GPUs depending on platform.
Deploy on iOS and iPadOS via Metal and on Android via OpenCL on Adreno and Mali GPUs.
The platform matrix includes web browsers via WebGPU and WASM, with the related WebLLM project for in-browser use.
Get the source code from GitHub and enter the project directory.
git clone https://github.com/mlc-ai/mlc-llm.git
cd mlc-llmThe README links to platform-specific install docs rather than one universal command, so follow the guide for your OS and GPU.
# Follow the installation guide for your platform - see the docs link in the READMEThe documentation's quick start walks through downloading a model and chatting with it via MLCEngine.
# Follow the quick start in the docs linked from the READMEIt is developed by the MLC AI community (the team behind TVM's machine learning compilation work) on GitHub.
Yes - MLC LLM is Apache-2.0 licensed and free to use, including commercially, under that license's terms.
No paid API key is required: MLC LLM runs open models locally on your own hardware. You only need model weights, which are downloaded from sources such as Hugging Face.
Compared with Ollama, which focuses on easy local serving, MLC LLM is a compiler stack that targets many more platforms - mobile and browsers included. Compared with vLLM, it trades a server-only focus for cross-platform reach.
Clone the repo, then follow the installation and quick start pages linked from the README for your platform, and run the documented first chat example.
TokenSpeed — a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability
Lightweight inference engine for running LLMs efficiently on any device.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Distribute and run LLMs with a single file.
Discover and run local AI apps, serving optimized LLMs from your own GPU and NPU.
Let any LLM actually watch a video — frames plus transcript, run locally.
Die anderen Projekte der Kategorie Lokale LLMs mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls mlc-llm nicht das Richtige ist.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Verwandte Projekte der Kategorie Lokale LLMs im Katalog, die mlc-llm bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
An open-source RAG-based tool for chatting with your documents.
🔥 MaxKB is an open-source platform for building enterprise-grade agents. 强大易用的开源企业级智能体平台。
Distribute and run LLMs with a single file.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Open-source 24/7 Cowork app for OpenClaw, Hermes, Claude Code, Codex, OpenCode and 20+ more CLI Agent | Customize your assistants | Team them up｜Star if you …
High-performance In-browser LLM Inference Engine
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.