Repolode
Submit
orailnoor

orailnoor/cross-platform-llm-client

A Flutter chat client that runs GGUF models on-device on Android and iOS, or switches to cloud APIs from OpenAI, Anthropic, Gemini, or Kimi.

Open-source alternative to:ChatGPT

View on GitHub
Stars
1,642
Forks
340
Open issues
23
Last commit
3 days ago

What is it?

PrivateLM is a cross-platform AI chat client built with Flutter. It runs local LLM inference on Android and iOS using llama.cpp wrapped in a custom Flutter plugin, with GPU acceleration through Vulkan on Android and Metal on iOS. Users can download GGUF models to the device and chat without an internet connection after the download completes.

The same interface can also connect to cloud providers, including OpenAI, Anthropic, Google Gemini, and Kimi (Moonshot AI). Cloud calls are normalized across the different API formats. On the web build, only cloud mode is available. Chats and settings are stored locally with Hive, and API keys are kept on the device and sent only to the chosen provider.

On first launch, the app checks device RAM and suggests a context size and token limit. Multimodal chat supports text and images, with vision via local Qwen2-VL models or supported cloud providers. A task view for multi-step workflows is still in development.

Key features

  • 01Runs GGUF models on Android and iOS with GPU acceleration.
  • 02Switches to OpenAI, Anthropic, Google Gemini, or Kimi APIs.
  • 03Supports text and image input, including local Qwen2-VL vision models.
  • 04Stores chats and settings locally with Hive.
  • 05Recommends context size and token limits based on device RAM.

Use cases

  • Chatting with a local model on a phone without sending data to a server.
  • Using one client to switch between on-device models and cloud APIs.
  • Sideloading a private AI chat app on iPhone or iPad via AltStore.

Quick start

  1. 1

    Install Flutter dependencies

    flutter pub get
  2. 2

    Build the Android debug APK

    flutter build apk --debug
  3. 3

    Build the web app

    flutter build web --release

Things to know

  • Local on-device inference is Android and iOS only; the web build is cloud-only.
  • Building for iOS requires a Mac and installing via AltStore, and sideloaded apps expire after 7 days.
  • Release Android builds require creating your own signing key configuration.

In collections

Run LLMs locallyPrivacy & local-first tools

Topics

#flutter#llm#on-device-ai#llama-cpp#gguf#offline#multimodal#chat-client

FAQ

What is cross-platform-llm-client?
A Flutter chat client that runs GGUF models on-device on Android and iOS, or switches to cloud APIs from OpenAI, Anthropic, Gemini, or Kimi. PrivateLM is a cross-platform AI chat client built with Flutter. It runs local LLM inference on Android and iOS using llama.cpp wrapped in a custom Flutter plugin, with GPU acceleration through Vulkan on Android and Metal on iOS. Users can download GGUF models to the device and chat without an internet connection after the download completes.
Is cross-platform-llm-client free to use? What license does it have?
cross-platform-llm-client is open source under the MIT license. Check the license terms before using it in commercial products.
Is cross-platform-llm-client still maintained?
Actively maintained. The last commit to cross-platform-llm-client was on Nov 4, 2026.
What can I use cross-platform-llm-client for?
Chatting with a local model on a phone without sending data to a server. Using one client to switch between on-device models and cloud APIs. Sideloading a private AI chat app on iPhone or iPad via AltStore.
How do I install cross-platform-llm-client?
Install Flutter dependencies: flutter pub get Build the Android debug APK: flutter build apk --debug Build the web app: flutter build web --release
Where can I get cross-platform-llm-client?
The source code and installation instructions are on GitHub: https://github.com/orailnoor/cross-platform-llm-client

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects