Open-source projects tagged “inference”

Python inference library for Kokoro-82M, an open-weight text-to-speech model with 82 million parameters.

A curated awesome-list of platforms, inference engines, UIs, models, tools and tutorials for running LLMs on your own hardware.

Python library that detects LLM decoding corruption such as repetition loops and language drift mid-stream, so bad tokens can be cut before users see them.

A minimal C11 LLM inference engine that runs 1B-class LLaMA-architecture GGUF models on low-cost boards like Raspberry Pi with about 45MB of RAM.

A curated Awesome-style list of platforms, tools, models and tutorials for running large language models locally.

Open source inference engine that tunes its kernels to your hardware to run each open model as fast as possible.

A learn-by-building ML course: a from-scratch NumPy deep learning framework (CLI) plus 350+ browser-based coding questions.