LLM local
can-i-finetune-this
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
★ 792 estrellasMIT
Ver la guía →airllm es un proyecto de LLM local de código abierto de lyogavin (lyogavin/airllm en GitHub). Tiene 35.511 estrellas en GitHub y 3.753 bifurcaciones, se publica bajo la licencia Apache-2.0 y se actualizó por última vez en GitHub el 8 de octubre de 2026.
Run a 70B LLM on a single 4GB GPU — layer-by-layer inference with almost no memory tricks skipped
Decide what you want to achieve with airllm and check the README on GitHub for requirements before installing.
Clone the repository from GitHub (or take the packaged install route in step 3 if the README offers one).
Follow the project's documented install commands — the Run section below lists the exact ones — and add any required API keys or models.
Work through the README's quickstart example end to end, then adapt it to your own project or data.
Layer-by-layer loading lets models far larger than your VRAM still run, trading speed for reach.
Pass a Hugging Face repo id (or a local path) and AirLLM handles the rest — the quickstart is a few lines of Python.
Model compression via bitsandbytes can speed inference up to 3x, per the README.
Works with popular open model families listed in the README, including Llama and other Hugging Face models.
Includes a training path with dataset preparation and a Python API, not just inference.
Documents running on Apple hardware as well as CUDA GPUs.
Get the AirLLM source, examples and notebooks.
git clone https://github.com/lyogavin/airllm.git
cd airllmInstall AirLLM (and bitsandbytes if you want compression) from PyPI.
pip install airllm
pip install -U bitsandbytesLoad a Hugging Face model id through AirLLM's Python API as shown in the README quickstart.
# from airllm import AutoModel
# model = AutoModel.from_pretrained('meta-llama/Llama-2-70b-hf') # see README quickstartairllm is maintained by the lyogavin organization/account on GitHub, together with open-source contributors, in the public repository lyogavin/airllm.
Yes — the code is open source under the Apache-2.0 license. You can use, modify and redistribute it under that license's terms.
No API key and no paid service — models download from Hugging Face (some are gated and need a free Hugging Face account/token and license acceptance).
Those quantize models to fit memory and are much faster; AirLLM instead runs full-precision huge models layer-by-layer — slower, but it can run models that would not fit even quantized.
Clone the repository from GitHub, follow the install commands in the Run section (taken from the project's README), and complete the documented quickstart example before adapting it to your own use case.
Check if your GPU can fine-tune an LLM — memory estimates and LoRA recipes.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
Distribute and run LLMs with a single file.
Compile and run LLMs anywhere - phones, laptops, browsers and GPUs - with one engine
Discover and run local AI apps, serving optimized LLMs from your own GPU and NPU.
Los demás proyectos de LLM local con más estrellas del catálogo de LibreAI: la lista realista de alternativas si airllm no es lo que buscas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Proyectos de LLM local cercanos en el catálogo, los más próximos en estrellas a airllm: sigue explorando a ambos lados.
Hundreds of models & providers. One command to find what runs on your hardware.
Strata — one-click Qwen3.8-Flash-Next inference on consumer hardware: chat, code and images, locally, no account, MIT
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Open-source 24/7 Cowork app for OpenClaw, Hermes, Claude Code, Codex, OpenCode and 20+ more CLI Agent | Customize your assistants | Team them up｜Star if you …
BitNet — Microsoft's official inference framework for 1-bit large language models, running fast on ordinary CPUs, MIT
Distribute and run LLMs with a single file.
Gratis y abierto: consigue el código fuente, dale una estrella al repositorio y empieza a crear.