Локальные LLM
can-i-finetune-this
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
★ 792 звездыMIT
Смотреть руководство →airllm — проект с открытым исходным кодом из категории «Локальные LLM» от lyogavin (lyogavin/airllm на GitHub). У него 35 511 звёзд на GitHub и 3 753 форка, он распространяется по лицензии Apache-2.0 и последний раз обновлялся на GitHub 8 октября 2026.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Decide what you want to achieve with airllm and check the README on GitHub for requirements before installing.
Clone the repository from GitHub (or take the packaged install route in step 3 if the README offers one).
Follow the project's documented install commands — the Run section below lists the exact ones — and add any required API keys or models.
Work through the README's quickstart example end to end, then adapt it to your own project or data.
Layer-by-layer loading lets models far larger than your VRAM still run, trading speed for reach.
Pass a Hugging Face repo id (or a local path) and AirLLM handles the rest — the quickstart is a few lines of Python.
Model compression via bitsandbytes can speed inference up to 3x, per the README.
Works with popular open model families listed in the README, including Llama and other Hugging Face models.
Includes a training path with dataset preparation and a Python API, not just inference.
Documents running on Apple hardware as well as CUDA GPUs.
Get the AirLLM source, examples and notebooks.
git clone https://github.com/lyogavin/airllm.git
cd airllmInstall AirLLM (and bitsandbytes if you want compression) from PyPI.
pip install airllm
pip install -U bitsandbytesLoad a Hugging Face model id through AirLLM's Python API as shown in the README quickstart.
# from airllm import AutoModel
# model = AutoModel.from_pretrained('meta-llama/Llama-2-70b-hf') # see README quickstartairllm is maintained by the lyogavin organization/account on GitHub, together with open-source contributors, in the public repository lyogavin/airllm.
Yes — the code is open source under the Apache-2.0 license. You can use, modify and redistribute it under that license's terms.
No API key and no paid service — models download from Hugging Face (some are gated and need a free Hugging Face account/token and license acceptance).
Those quantize models to fit memory and are much faster; AirLLM instead runs full-precision huge models layer-by-layer — slower, but it can run models that would not fit even quantized.
Clone the repository from GitHub, follow the install commands in the Run section (taken from the project's README), and complete the documented quickstart example before adapting it to your own use case.
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
Distribute and run LLMs with a single file.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
Discover and run local AI apps, serving optimized LLMs from your own GPU and NPU.
Другие проекты из категории «Локальные LLM» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если airllm вам не подходит.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Соседние проекты из категории «Локальные LLM» в каталоге, ближайшие по числу звёзд к airllm — продолжайте изучать полку по обе стороны от него.
Hundreds of models & providers. One command to find what runs on your hardware.
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Open-source 24/7 Cowork app for OpenClaw, Hermes, Claude Code, Codex, OpenCode and 20+ more CLI Agent | Customize your assistants | Team them up｜Star if you …
BitNet — Microsoft's official inference framework for 1-bit large language models, running fast on ordinary CPUs, MIT
Distribute and run LLMs with a single file.
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.