Local LLM
chatgpt-subtitle-translator
Token efficient translation utility built on ChatGPT or any OpenAI compatible LLM.
★ 389 starsMIT
View guide →StepFun's open 196B MoE model that activates 11B parameters per token, built for fast agentic reasoning.
Step-3.5-Flash is an open-source local llm project by stepfun-ai (stepfun-ai/Step-3.5-Flash on GitHub). It has 2,083 GitHub stars and 87 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 3 April 2026.
Decide how you will access the model: hosted API (OpenRouter or StepFun), or local deployment on high-end hardware.
For local use, clone the repository to get its deployment patches, Dockerfiles, and guides.
Install a supported inference backend such as vLLM (nightly or the patched 0.15.1 build) or SGLang, and download the weights from Hugging Face or ModelScope.
Launch the server with the documented Step 3.5 parsers and expert-parallel settings, then call its OpenAI-compatible endpoint from your agent or app.
288 routed experts per layer with top-8 selection give the knowledge of a 196B model at roughly 11B active parameters per token.
The MTP-3 head predicts multiple tokens per forward pass, which the project credits for its high generation throughput in typical and coding workloads.
A 3:1 mix of sliding-window and full-attention layers keeps long-context cost down across large codebases and datasets.
Tuned and evaluated for long-horizon agent tasks and software engineering, with published results on benchmarks such as SWE-bench Verified and Terminal-Bench 2.0 in the README.
Documented serving with vLLM, SGLang, Transformers for verification, and llama.cpp, plus fp8 and bf16 variants.
Available through OpenRouter (including a free tier model id) and StepFun's API, with integration guides for OpenClaw, Claude Code, and Roo Code.
The repo holds deployment patches, Dockerfiles, and cookbooks; weights come from Hugging Face or ModelScope.
git clone https://github.com/stepfun-ai/Step-3.5-Flash.git
cd Step-3.5-FlashThe README recommends the vLLM nightly build, or the patched 0.15.1 image for full parser support.
pip install -U vllm --pre --index-url https://pypi.org/simple --extra-index-url https://wheels.vllm.ai/nightlyLaunch vLLM with the Step 3.5 reasoning and tool-call parsers and expert parallelism.
vllm serve --served-model-name step3p5-flash --tensor-parallel-size 8 --enable-expert-parallel --reasoning-parser step3p5 --tool-call-parser step3p5 --trust-remote-code # add the model path/checkpoint per the READMEIt is developed and released by StepFun, the company behind the Step model series.
The model is Apache-2.0 licensed and the weights are openly downloadable. Hosted access has free and paid tiers via OpenRouter and StepFun; local serving costs your own hardware.
Only for hosted access: OpenRouter or StepFun API keys are needed there. Local deployment with vLLM or SGLang needs no API key.
Dense models activate every parameter per token; Step-3.5-Flash activates about 11B of 196B, trading total size for much lower decoding cost at comparable reported capability.
For a quick trial, use the OpenRouter model id stepfun/step-3.5-flash:free with the openai Python SDK; for local use, follow the repo's vLLM deployment section.
Token efficient translation utility built on ChatGPT or any OpenAI compatible LLM.
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
Cloud-native runtime for building and running agentic AI in production.
Go-built AI gateway unifying many providers with routing and guardrails.
DeepSeek-Coder-V2 — DeepSeek's open code model family that challenged closed-source code models.
Local-first widget tracking token usage and costs across 43+ AI coding tools.
The most-starred other Local LLM projects in the LibreAI catalogue — the realistic shortlist if Step-3.5-Flash is not the right fit.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Free and open — grab the source, star the repo, and start creating.