Repolode
Submit
qualcomm

qualcomm/GenieX

On-device LLM and VLM inference runtime for Qualcomm Snapdragon devices, running models on the NPU, GPU, or CPU via CLI, Python, Android, Docker, or an OpenAI-compatible server.

View on GitHubWebsite
Stars
8,450
Forks
1,070
Open issues
71
Last commit
5 hours ago

What is it?

GenieX is an on-device generative AI inference runtime for Qualcomm Snapdragon hardware. It runs GGUF models from Hugging Face, ModelScope, or Docker Hub through llama.cpp on the Hexagon NPU, Adreno GPU, or CPU, and runs pre-compiled bundles from Qualcomm AI Hub on the NPU through the Qualcomm AI Engine Direct runtime.

The project wraps one C SDK with several interfaces: a geniex command-line tool, a Python package whose AutoModelForCausalLM API mirrors Hugging Face transformers, an Android Kotlin/Java SDK, a Docker image, and a local OpenAI-compatible HTTP server. Windows ARM64, Android, and Linux ARM64 are supported, and the project only targets Qualcomm Snapdragon devices.

Key features

  • 01Runs GGUF models on Hexagon NPU, Adreno GPU, or CPU via llama.cpp.
  • 02Runs pre-compiled Qualcomm AI Hub bundles on the NPU.
  • 03Provides a CLI, Python API, Android SDK, and Docker image.
  • 04Serves an OpenAI-compatible API at a local endpoint.
  • 05Supports both LLMs and VLMs.

Use cases

  • Running a local chat or assistant on a Snapdragon laptop without cloud calls.
  • Adding on-device LLM features to an Android app with the Kotlin/Java SDK.
  • Pointing existing OpenAI-client code at a locally served model on an ARM Linux IoT device.
  • Benchmarking GGUF models on Qualcomm NPUs versus CPU or GPU.

Quick start

  1. 1

    Install the Python package

    pip install geniex
  2. 2

    Run the Python example to chat with a model

  3. 3

    Or chat from the command line

    geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf

Things to know

  • Runs only on Qualcomm Snapdragon devices.
  • Status is Developer Preview, so interfaces may change.
  • The Linux CLI install relies on a curl-to-shell script, which the README provides and which you must run manually.

Topics

#local-llm#on-device-ai#npu#qualcomm#snapdragon#llama-cpp#openai-compatible#inference#gemma4#go

FAQ

What is GenieX?
On-device LLM and VLM inference runtime for Qualcomm Snapdragon devices, running models on the NPU, GPU, or CPU via CLI, Python, Android, Docker, or an OpenAI-compatible server. GenieX is an on-device generative AI inference runtime for Qualcomm Snapdragon hardware. It runs GGUF models from Hugging Face, ModelScope, or Docker Hub through llama.cpp on the Hexagon NPU, Adreno GPU, or CPU, and runs pre-compiled bundles from Qualcomm AI Hub on the NPU through the Qualcomm AI Engine Direct runtime.
Is GenieX free to use? What license does it have?
GenieX is open source under the BSD-3-Clause license. Check the license terms before using it in commercial products.
Is GenieX still maintained?
Actively maintained. The last commit to GenieX was on Oct 21, 2026.
What can I use GenieX for?
Running a local chat or assistant on a Snapdragon laptop without cloud calls. Adding on-device LLM features to an Android app with the Kotlin/Java SDK. Pointing existing OpenAI-client code at a locally served model on an ARM Linux IoT device. Benchmarking GGUF models on Qualcomm NPUs versus CPU or GPU.
How do I install GenieX?
Install the Python package: pip install geniex Run the Python example to chat with a model Or chat from the command line: geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf
Where can I get GenieX?
The source code and installation instructions are on GitHub: https://github.com/qualcomm/GenieX

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects