Repolode
Submit
mybigday

mybigday/llama.rn

React Native binding of llama.cpp for running GGUF LLMs on-device in iOS and Android apps.

View on GitHub
Stars
1,059
Forks
132
Open issues
23
Last commit
3 hours ago

What is it?

llama.rn wraps llama.cpp so React Native apps can load GGUF language models and run inference directly on the phone, without a server. It exposes JavaScript APIs for chat and text completion, embeddings, reranking, tokenization, and session save/load.

The library covers more than plain text generation. It supports multimodal models through mmproj projectors for image and audio input, tool calling via Jinja chat templates, and structured output using GBNF grammars or JSON schemas. Parallel slot-based request handling and MTP speculative decoding are also included. GPU acceleration uses Metal on iOS and OpenCL on supported Qualcomm Adreno devices, with experimental Hexagon NPU support on newer Snapdragon chips.

An experimental text-to-speech path runs several neural TTS model families through the codec.cpp backend, with voice cloning from reference audio. Starting with v0.10, the library requires React Native's New Architecture; older versions are on a separate branch.

Key features

  • 01Runs GGUF models on iOS and Android through React Native JavaScript APIs.
  • 02Supports Metal GPU acceleration on iOS and OpenCL on supported Qualcomm devices.
  • 03Handles vision and audio input for multimodal models via mmproj files.
  • 04Provides tool calling and structured output through Jinja templates and GBNF or JSON Schema grammars.
  • 05Offers slot-based parallel completions, embeddings, and reranking.

Use cases

  • Building a privacy-focused chatbot that runs fully offline on a phone.
  • Adding on-device document search with embeddings and reranking in a mobile app.
  • Prototyping vision or audio understanding features in a React Native app without a backend.

Quick start

  1. 1

    Install the package from npm

    npm install llama.rn
  2. 2

    Re-run pod install for iOS

    npx pod-install
  3. 3

    Load a GGUF model and run completion using initLlama

Things to know

  • Starting with v0.10 it requires React Native's New Architecture; Old Architecture needs the v0.9 branch.
  • Native binaries are downloaded during postinstall from GitHub releases, so installs need network access and may need a manual download step.
  • GPU and NPU acceleration (OpenCL, Hexagon) are limited to specific Qualcomm devices, and Hexagon is experimental.
  • Text-to-speech support is experimental.

Topics

#react-native#llama-cpp#on-device-ai#llm-inference#gguf#mobile#multimodal#ios#android#llama

FAQ

What is llama.rn?
React Native binding of llama.cpp for running GGUF LLMs on-device in iOS and Android apps. llama.rn wraps llama.cpp so React Native apps can load GGUF language models and run inference directly on the phone, without a server. It exposes JavaScript APIs for chat and text completion, embeddings, reranking, tokenization, and session save/load.
Is llama.rn free to use? What license does it have?
llama.rn is open source under the MIT license. Check the license terms before using it in commercial products.
Is llama.rn still maintained?
Actively maintained. The last commit to llama.rn was on Oct 21, 2026.
What can I use llama.rn for?
Building a privacy-focused chatbot that runs fully offline on a phone. Adding on-device document search with embeddings and reranking in a mobile app. Prototyping vision or audio understanding features in a React Native app without a backend.
How do I install llama.rn?
Install the package from npm: npm install llama.rn Re-run pod install for iOS: npx pod-install Load a GGUF model and run completion using initLlama
Where can I get llama.rn?
The source code and installation instructions are on GitHub: https://github.com/mybigday/llama.rn

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects