Lokale LLMs
nobodywho
Lightweight inference engine for running LLMs efficiently on any device.
★ 1,5 Tsd. SterneEUPL-1.2
Anleitung ansehen →MNN ist ein Open-Source-Projekt aus der Kategorie Lokale LLMs von alibaba (alibaba/MNN auf GitHub). Es hat 16.192 Sterne auf GitHub und 2.467 Forks, wird unter der Lizenz Apache-2.0 veröffentlicht und wurde zuletzt am 8. Oktober 2026 auf GitHub aktualisiert.
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Decide whether you are targeting Android, iOS, embedded Linux or desktop, and choose a model to convert, such as a Qwen-series LLM for on-device chat.
Get the MNN source, which includes the converter, runtime and sample apps including the MNN LLM chat apps.
Convert your TensorFlow, ONNX, Caffe or TorchScript model to MNN format and build the library for your platform, choosing CPU or GPU backends.
Integrate the runtime into your app or run the included chat app, then tune quantization (FP16/INT8) for size and speed.
Optimized for devices with no dependencies: the Android core library is about 800KB, and FP16/INT8 quantization can cut model size by 50-70%.
The converter supports models from TensorFlow, Caffe, ONNX and TorchScript, covering CNN, RNN, GAN and Transformer networks.
Runs on ARM and x64 CPUs with optimized assembly, plus Metal, OpenCL and Vulkan on mobile GPUs, CUDA on NVIDIA, and a Hexagon DSP backend.
Powers local LLM apps, with recent releases adding support for Qwen3 and Qwen3-VL series models and multimodal chat on Android and iOS.
An MNN Python API lets ML engineers run inference, training and image processing without writing C++.
MNN supports training models on PC or mobile in addition to inference, plus a lightweight OpenCV-like image processing module.
Clone the MNN source, including the converter, runtimes and sample apps.
git clone https://github.com/alibaba/MNN.git
cd MNNConvert an ONNX, TensorFlow, Caffe or TorchScript model into MNN format with the included converter before deploying it.
# Convert your model with the MNN converter - see README for the exact commandBuild MNN for Android, iOS or desktop following the platform guide, or open one of the included LLM chat sample apps.
# Build for your target platform - see README for the exact commandMNN is developed by Alibaba and the open-source community; it is battle-tested in Alibaba's own products.
Yes. It is released under the Apache-2.0 license, which allows commercial use.
No. MNN runs models locally on your device, so there is no cloud API or key involved.
MNN is a lightweight C++ engine focused on fast on-device inference, with its own converter for TensorFlow, Caffe, ONNX and TorchScript models and strong mobile GPU and LLM support; the best choice depends on your model and target hardware.
Clone the repo, convert a model with the MNN converter, and try one of the included sample apps, such as the Android or iOS LLM chat app.
Lightweight inference engine for running LLMs efficiently on any device.
A high-throughput and memory-efficient inference and serving engine for LLMs
Offline inference engine for art.
Fast Rust scraper that turns web pages into clean data for LLMs.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Die anderen Projekte der Kategorie Lokale LLMs mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls MNN nicht das Richtige ist.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Verwandte Projekte der Kategorie Lokale LLMs im Katalog, die MNN bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
High-performance In-browser LLM Inference Engine
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Access large language models from the command-line
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.