Local LLM
litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, an…
★ 60.1k starsCustom
View guide →Google DeepMind's Gemma open-weight LLM library — install, load and chat with Gemma models in Python.
gemma is an open-source local llm project by google-deepmind (google-deepmind/gemma on GitHub). It has 5,774 GitHub stars and 1,032 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 6 October 2026.
Decide which Gemma model size fits your hardware and whether you need text-only or multimodal input.
Install the gemma package from PyPI into your Python environment.
Get the model weights from the official Gemma download channels and note their license terms.
Load the model with the example code and start a multi-turn chat.
The reference Python library for Gemma models, maintained by Google DeepMind.
The bundled example runs a multi-turn, multimodal conversation with Gemma 4-class models.
pip install gemma gets the library; weights are fetched separately so code and models update independently.
Links to the full documentation site and the technical reports describing model capabilities and metrics.
Points to other Gemma implementations and tooling across frameworks if you do not run JAX.
Gemma models are open-weight releases from Google, usable locally under the Gemma terms.
Clone the Gemma library repository.
git clone https://github.com/google-deepmind/gemma.git
cd gemmaInstall the gemma package from PyPI.
pip install gemmaDownload the weights as the README directs, then run the minimal multi-turn conversation example.
python -m gemma.examples # see README for the exact commandGoogle DeepMind maintains the gemma repository and the Gemma model family.
The library is Apache-2.0 licensed; the model weights are open weights distributed under Google's Gemma terms, which you accept when downloading them.
No — you run the weights locally. You do need to download the weights through the official channels, which may require accepting the Gemma terms.
This is DeepMind's own JAX-based reference implementation; the Transformers route is often easier if your stack is PyTorch, but this repo is the canonical source for the library and examples.
pip install gemma, download the weights per the README, and run the conversation example.
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, an…
GPT4All — run local LLMs on any device: a free desktop chat client and Python SDK that run open models entirely on your CPU, no GPU or cloud required
A no-install GUI for Ollama and other local LLM providers.
Open bilingual (Chinese-English) 6B chat LLM with 32K-context variant and low-memory INT4 inference.
Access large language models from the command-line
KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install
The most-starred other Local LLM projects in the LibreAI catalogue — the realistic shortlist if gemma is not the right fit.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Free and open — grab the source, star the repo, and start creating.