LLM locaux
nobodywho
Lightweight inference engine for running LLMs efficiently on any device.
★ 1,5 k étoilesEUPL-1.2
Voir le guide →MNN est un projet de LLM locaux open source, créé par alibaba (alibaba/MNN sur GitHub). Il compte 16 192 étoiles sur GitHub et 2 467 forks, il est publié sous la licence Apache-2.0 et a été mis à jour pour la dernière fois sur GitHub le 8 octobre 2026.
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Decide whether you are targeting Android, iOS, embedded Linux or desktop, and choose a model to convert, such as a Qwen-series LLM for on-device chat.
Get the MNN source, which includes the converter, runtime and sample apps including the MNN LLM chat apps.
Convert your TensorFlow, ONNX, Caffe or TorchScript model to MNN format and build the library for your platform, choosing CPU or GPU backends.
Integrate the runtime into your app or run the included chat app, then tune quantization (FP16/INT8) for size and speed.
Optimized for devices with no dependencies: the Android core library is about 800KB, and FP16/INT8 quantization can cut model size by 50-70%.
The converter supports models from TensorFlow, Caffe, ONNX and TorchScript, covering CNN, RNN, GAN and Transformer networks.
Runs on ARM and x64 CPUs with optimized assembly, plus Metal, OpenCL and Vulkan on mobile GPUs, CUDA on NVIDIA, and a Hexagon DSP backend.
Powers local LLM apps, with recent releases adding support for Qwen3 and Qwen3-VL series models and multimodal chat on Android and iOS.
An MNN Python API lets ML engineers run inference, training and image processing without writing C++.
MNN supports training models on PC or mobile in addition to inference, plus a lightweight OpenCV-like image processing module.
Clone the MNN source, including the converter, runtimes and sample apps.
git clone https://github.com/alibaba/MNN.git
cd MNNConvert an ONNX, TensorFlow, Caffe or TorchScript model into MNN format with the included converter before deploying it.
# Convert your model with the MNN converter - see README for the exact commandBuild MNN for Android, iOS or desktop following the platform guide, or open one of the included LLM chat sample apps.
# Build for your target platform - see README for the exact commandMNN is developed by Alibaba and the open-source community; it is battle-tested in Alibaba's own products.
Yes. It is released under the Apache-2.0 license, which allows commercial use.
No. MNN runs models locally on your device, so there is no cloud API or key involved.
MNN is a lightweight C++ engine focused on fast on-device inference, with its own converter for TensorFlow, Caffe, ONNX and TorchScript models and strong mobile GPU and LLM support; the best choice depends on your model and target hardware.
Clone the repo, convert a model with the MNN converter, and try one of the included sample apps, such as the Android or iOS LLM chat app.
Lightweight inference engine for running LLMs efficiently on any device.
A high-throughput and memory-efficient inference and serving engine for LLMs
Offline inference engine for art.
Fast Rust scraper that turns web pages into clean data for LLMs.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Les autres projets de la catégorie LLM locaux avec le plus d'étoiles du catalogue LibreAI — la sélection réaliste d'alternatives si MNN ne vous convient pas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projets de la catégorie LLM locaux proches dans le catalogue, les plus proches en étoiles de MNN — continuez à explorer de part et d'autre.
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
High-performance In-browser LLM Inference Engine
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Access large language models from the command-line
Gratuit et ouvert : récupérez le code source, mettez une étoile au dépôt et commencez à créer.