#local-inference

Open-source projects tagged “local-inference”

Curated GitHub repositories related to local-inference, with AI summaries and key stats.

Sort:Recently added

4 repositories

Niko1221 / Strata

Runs the 125B-parameter Qwen3.8-Flash-Next model on a consumer NVIDIA or AMD GPU PC, exposing OpenAI- and Anthropic-compatible APIs on localhost.

AI & Machine Learning #llm #local-inference #gpu #openai-api #quantization

27.4K2.9KC++MITUpdated 2 days ago

Comfy-Org / ComfyUI

GitHub

A node-graph GUI, API and backend for running diffusion models to generate images, video, audio and 3D locally.

AI & Machine Learning #stable-diffusion #diffusion-models #image-generation #video-generation #node-editor

138.5K16.5KPythonGPL-3.0Updated 3 hours ago

12britz / awesome-free-models

GitHub

A community-maintained list of AI models, APIs, and tools that can be used without paying, covering free tiers, open-weight models, and local inference tools.

Learning Resources #ai #llm #free-api #open-weight-models #local-inference

2.5K244OtherUpdated 7 days ago

ekwek1 / soprano

GitHub

Ultra-lightweight on-device text-to-speech model with an 80M-parameter architecture, streaming output at 32kHz, and CPU, CUDA and MPS support.

AI & Machine Learning #text-to-speech #tts #speech-synthesis #streaming #onnx

1.6K138PythonApache-2.0Updated 9 months ago