Repolode
Submit
RunanywhereAI

RunanywhereAI/runanywhere-sdks

Cross-platform SDK for running LLMs, speech, vision and image models fully on-device across iOS, Android, web, desktop and servers.

View on GitHubWebsite
Stars
10,332
Forks
410
Open issues
210
Last commit
3 days ago

What is it?

RunAnywhere is a C++ core with bindings for Swift, Kotlin, Flutter, React Native, Web, Electron and Python. It lets an app run language models, vision-language models, speech-to-text, text-to-speech, voice activity detection, embeddings, RAG and diffusion-based image generation on the device itself, without sending data to a server.

A capability registry routes each call to the highest-priority engine that supports the modality on the current hardware. Supported engines include QHexRT for the Snapdragon Hexagon NPU, MLX on Apple silicon, llama.cpp with Metal, CUDA or WebGPU backends, sherpa with ONNX for speech and embeddings, and Core ML for diffusion. The same application code works across these targets.

The project also ships a Python OpenAI-compatible local server, a terminal CLI called wally, and an optional console for deploying models and collecting per-device telemetry. Several SDKs are still in beta, preview or alpha, and some features such as wake-word detection are not implemented.

Key features

  • 01Runs LLM, VLM, speech, TTS, embeddings and diffusion models locally on device.
  • 02Routes inference to engines such as QHexRT, MLX, llama.cpp, sherpa/ONNX and Core ML based on hardware.
  • 03Offers the same API across Swift, Kotlin, Flutter, React Native, Web, Electron and Python.
  • 04Supports structured JSON output, tool calling and RAG over local documents.
  • 05Provides an OpenAI-compatible local server in the Python SDK and wally CLI.

Use cases

  • Building an offline voice assistant that runs speech recognition, a local LLM and speech synthesis on a phone.
  • Adding private document Q&A to a desktop app without sending files to a cloud API.
  • Serving local models behind an OpenAI-compatible endpoint for existing client code.
  • Running an on-device vision-language model for camera-based description in a mobile app.

Quick start

  1. 1

    Install the Python package from PyPI

    pip install runanywhere
  2. 2

    Initialize the SDK and generate text with the README's Python example (the model downloads on first use)

Things to know

  • The license is not a standard open-source license and is listed as NOASSERTION, so check the LICENSE file before commercial use.
  • Some features depend on the platform or backend, and CUDA is an opt-in source build.
  • The Electron desktop binding is Windows-first and ships a prebuilt addon only for win32-x64.

In collections

Run LLMs locallyPrivacy & local-first toolsAndroid apps & phone tools

Topics

#on-device-ai#llm#edge-inference#voice-ai#multimodal#llama-cpp#mobile-ai#offline-first#android#apple-intelligence

FAQ

What is runanywhere-sdks?
Cross-platform SDK for running LLMs, speech, vision and image models fully on-device across iOS, Android, web, desktop and servers. RunAnywhere is a C++ core with bindings for Swift, Kotlin, Flutter, React Native, Web, Electron and Python. It lets an app run language models, vision-language models, speech-to-text, text-to-speech, voice activity detection, embeddings, RAG and diffusion-based image generation on the device itself, without sending data to a server.
Is runanywhere-sdks free to use? What license does it have?
runanywhere-sdks is open source under the Other license. Check the license terms before using it in commercial products.
Is runanywhere-sdks still maintained?
Actively maintained. The last commit to runanywhere-sdks was on Nov 5, 2026.
What can I use runanywhere-sdks for?
Building an offline voice assistant that runs speech recognition, a local LLM and speech synthesis on a phone. Adding private document Q&A to a desktop app without sending files to a cloud API. Serving local models behind an OpenAI-compatible endpoint for existing client code. Running an on-device vision-language model for camera-based description in a mobile app.
How do I install runanywhere-sdks?
Install the Python package from PyPI: pip install runanywhere Initialize the SDK and generate text with the README's Python example (the model downloads on first use)
Where can I get runanywhere-sdks?
The source code and installation instructions are on GitHub: https://github.com/RunanywhereAI/runanywhere-sdks

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects