Repolode
Submit
josStorer

josStorer/RWKV-Runner

A desktop tool that automates downloading, starting and managing RWKV large language models, exposing an OpenAI-compatible API.

Open-source alternative to:ChatGPT

View on GitHubWebsite
Stars
6,488
Forks
594
Open issues
180
Last commit
2 months ago

What is it?

RWKV Runner is a lightweight management and startup tool for RWKV, an open-source large language model. It handles model download, conversion, and loading with minimal setup, so users do not need to manually configure Python environments or dependencies. The main program is a small executable of a few megabytes.

The project exposes an interface compatible with the OpenAI API, so existing ChatGPT-style clients can be pointed at a local RWKV model by changing the API URL. It also ships with chat, completion, and composition interfaces, a built-in WebUI option, and VRAM presets for different hardware. A backend-python component can be deployed on a server, with the desktop app used as a client.

The repository includes a Python backend, a frontend, and deployment examples. Supported platforms include Windows, macOS, and Linux.

Key features

  • 01One-click startup and management of RWKV models with automatic dependency installation.
  • 02OpenAI API-compatible endpoints for use with existing ChatGPT clients.
  • 03Pre-set multi-level VRAM configurations, with WebGPU mode for AMD and Intel GPUs.
  • 04Built-in chat, completion, and composition interfaces, plus a model conversion tool.
  • 05Frontend and backend can be deployed separately, including a server-side WebUI option.

Use cases

  • Running a local RWKV model without manually setting up a Python environment.
  • Pointing existing OpenAI-compatible tools or chat clients at a self-hosted model.
  • Deploying the backend on a server and using the desktop app as a client.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/josStorer/RWKV-Runner
  2. 2

    Enter the project directory

    cd RWKV-Runner
  3. 3

    Start the backend inference service (then load a model via the /switch-model API)

    python ./backend-python/main.py

Things to know

  • Windows Defender may flag the executable as a virus, requiring manual trust exceptions.
  • LoRA finetuning is available on Windows only.
  • Running large RWKV models requires substantial VRAM or a compatible GPU setup.

Topics

#llm#rwkv#openai-api#local-llm#model-management#desktop-app#api#api-client#chatgpt#tool

FAQ

What is RWKV-Runner?
A desktop tool that automates downloading, starting and managing RWKV large language models, exposing an OpenAI-compatible API. RWKV Runner is a lightweight management and startup tool for RWKV, an open-source large language model. It handles model download, conversion, and loading with minimal setup, so users do not need to manually configure Python environments or dependencies. The main program is a small executable of a few megabytes.
Is RWKV-Runner free to use? What license does it have?
RWKV-Runner is open source under the MIT license. Check the license terms before using it in commercial products.
Is RWKV-Runner still maintained?
Updated within 6 months. The last commit to RWKV-Runner was on Sep 4, 2026.
What can I use RWKV-Runner for?
Running a local RWKV model without manually setting up a Python environment. Pointing existing OpenAI-compatible tools or chat clients at a self-hosted model. Deploying the backend on a server and using the desktop app as a client.
How do I install RWKV-Runner?
Clone the repository: git clone https://github.com/josStorer/RWKV-Runner Enter the project directory: cd RWKV-Runner Start the backend inference service (then load a model via the /switch-model API): python ./backend-python/main.py
Where can I get RWKV-Runner?
The source code and installation instructions are on GitHub: https://github.com/josStorer/RWKV-Runner

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects