Локальные LLM
nobodywho
Lightweight inference engine for running LLMs efficiently on any device.
★ 1,5 тыс. звёздEUPL-1.2
Смотреть руководство →MNN — проект с открытым исходным кодом из категории «Локальные LLM» от alibaba (alibaba/MNN на GitHub). У него 16 192 звезды на GitHub и 2 467 форков, он распространяется по лицензии Apache-2.0 и последний раз обновлялся на GitHub 8 октября 2026.
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Decide whether you are targeting Android, iOS, embedded Linux or desktop, and choose a model to convert, such as a Qwen-series LLM for on-device chat.
Get the MNN source, which includes the converter, runtime and sample apps including the MNN LLM chat apps.
Convert your TensorFlow, ONNX, Caffe or TorchScript model to MNN format and build the library for your platform, choosing CPU or GPU backends.
Integrate the runtime into your app or run the included chat app, then tune quantization (FP16/INT8) for size and speed.
Optimized for devices with no dependencies: the Android core library is about 800KB, and FP16/INT8 quantization can cut model size by 50-70%.
The converter supports models from TensorFlow, Caffe, ONNX and TorchScript, covering CNN, RNN, GAN and Transformer networks.
Runs on ARM and x64 CPUs with optimized assembly, plus Metal, OpenCL and Vulkan on mobile GPUs, CUDA on NVIDIA, and a Hexagon DSP backend.
Powers local LLM apps, with recent releases adding support for Qwen3 and Qwen3-VL series models and multimodal chat on Android and iOS.
An MNN Python API lets ML engineers run inference, training and image processing without writing C++.
MNN supports training models on PC or mobile in addition to inference, plus a lightweight OpenCV-like image processing module.
Clone the MNN source, including the converter, runtimes and sample apps.
git clone https://github.com/alibaba/MNN.git
cd MNNConvert an ONNX, TensorFlow, Caffe or TorchScript model into MNN format with the included converter before deploying it.
# Convert your model with the MNN converter - see README for the exact commandBuild MNN for Android, iOS or desktop following the platform guide, or open one of the included LLM chat sample apps.
# Build for your target platform - see README for the exact commandMNN is developed by Alibaba and the open-source community; it is battle-tested in Alibaba's own products.
Yes. It is released under the Apache-2.0 license, which allows commercial use.
No. MNN runs models locally on your device, so there is no cloud API or key involved.
MNN is a lightweight C++ engine focused on fast on-device inference, with its own converter for TensorFlow, Caffe, ONNX and TorchScript models and strong mobile GPU and LLM support; the best choice depends on your model and target hardware.
Clone the repo, convert a model with the MNN converter, and try one of the included sample apps, such as the Android or iOS LLM chat app.
Lightweight inference engine for running LLMs efficiently on any device.
A high-throughput and memory-efficient inference and serving engine for LLMs
Offline inference engine for art.
Fast Rust scraper that turns web pages into clean data for LLMs.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Другие проекты из категории «Локальные LLM» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если MNN вам не подходит.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Соседние проекты из категории «Локальные LLM» в каталоге, ближайшие по числу звёзд к MNN — продолжайте изучать полку по обе стороны от него.
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
High-performance In-browser LLM Inference Engine
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Access large language models from the command-line
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.