ローカルLLM
nobodywho
Lightweight inference engine for running LLMs efficiently on any device.
★ 1.5k スターEUPL-1.2
ガイドを見る →MNNは、alibabaが公開しているオープンソースのローカルLLMプロジェクトです（GitHub: alibaba/MNN）。GitHubスターは16,192、フォークは2,467で、Apache-2.0ライセンスの下で公開されています。GitHubでの最終更新は2026年10月8日です。
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
Decide whether you are targeting Android, iOS, embedded Linux or desktop, and choose a model to convert, such as a Qwen-series LLM for on-device chat.
Get the MNN source, which includes the converter, runtime and sample apps including the MNN LLM chat apps.
Convert your TensorFlow, ONNX, Caffe or TorchScript model to MNN format and build the library for your platform, choosing CPU or GPU backends.
Integrate the runtime into your app or run the included chat app, then tune quantization (FP16/INT8) for size and speed.
Optimized for devices with no dependencies: the Android core library is about 800KB, and FP16/INT8 quantization can cut model size by 50-70%.
The converter supports models from TensorFlow, Caffe, ONNX and TorchScript, covering CNN, RNN, GAN and Transformer networks.
Runs on ARM and x64 CPUs with optimized assembly, plus Metal, OpenCL and Vulkan on mobile GPUs, CUDA on NVIDIA, and a Hexagon DSP backend.
Powers local LLM apps, with recent releases adding support for Qwen3 and Qwen3-VL series models and multimodal chat on Android and iOS.
An MNN Python API lets ML engineers run inference, training and image processing without writing C++.
MNN supports training models on PC or mobile in addition to inference, plus a lightweight OpenCV-like image processing module.
Clone the MNN source, including the converter, runtimes and sample apps.
git clone https://github.com/alibaba/MNN.git
cd MNNConvert an ONNX, TensorFlow, Caffe or TorchScript model into MNN format with the included converter before deploying it.
# Convert your model with the MNN converter - see README for the exact commandBuild MNN for Android, iOS or desktop following the platform guide, or open one of the included LLM chat sample apps.
# Build for your target platform - see README for the exact commandMNN is developed by Alibaba and the open-source community; it is battle-tested in Alibaba's own products.
Yes. It is released under the Apache-2.0 license, which allows commercial use.
No. MNN runs models locally on your device, so there is no cloud API or key involved.
MNN is a lightweight C++ engine focused on fast on-device inference, with its own converter for TensorFlow, Caffe, ONNX and TorchScript models and strong mobile GPU and LLM support; the best choice depends on your model and target hardware.
Clone the repo, convert a model with the MNN converter, and try one of the included sample apps, such as the Android or iOS LLM chat app.
Lightweight inference engine for running LLMs efficiently on any device.
A high-throughput and memory-efficient inference and serving engine for LLMs
Offline inference engine for art.
Fast Rust scraper that turns web pages into clean data for LLMs.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
LibreAIカタログに掲載されたローカルLLMプロジェクトの中で最もスターの多いもの — MNNが合わなかった場合に検討したい、現実的な候補リストです。
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
カタログの中でMNNとスター数が近いローカルLLMプロジェクト — 前後の棚もそのまま見て回れます。
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
High-performance In-browser LLM Inference Engine
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
Access large language models from the command-line