Repolode
Submit
FluidInference

FluidInference/FluidAudio

A Swift SDK running speech models (ASR, TTS, VAD, speaker diarization) locally on Apple devices via CoreML and the Neural Engine.

Open-source alternative to:Apple Dictation

View on GitHubWebsite
Stars
3,271
Forks
504
Open issues
29
Last commit
5 days ago

What is it?

FluidAudio is a Swift SDK for running audio AI fully on-device on Apple platforms. Inference runs on the Apple Neural Engine through CoreML models, which the project says reduces memory use and CPU load compared with running on the CPU or GPU, making it suitable for background and always-on workloads.

The SDK covers automatic speech recognition using Parakeet TDT and Parakeet EOU models, text-to-speech with Kokoro, PocketTTS and Chatterbox models, voice activity detection based on Silero, and both online and offline speaker diarization. It also provides speaker embedding extraction and an echo cancellation and noise suppression component in beta. Models are published on Hugging Face, and the project includes a CLI for batch processing and benchmarks.

Integration is through Swift Package Manager or CocoaPods, with optional package traits to exclude subsystems an app does not use. Wrappers exist for React Native and Rust/Tauri, and the documentation covers custom model registry URLs, proxies and an offline-only mode.

Key features

  • 01Speech recognition with Parakeet TDT and EOU models, including streaming transcription.
  • 02Text-to-speech with Kokoro, PocketTTS and Chatterbox models.
  • 03Silero-based voice activity detection for offline and streaming use.
  • 04Online and offline speaker diarization plus speaker embedding extraction.
  • 05Inference on the Apple Neural Engine via CoreML, avoiding GPU and MPS.
  • 06Integration via Swift Package Manager with optional traits to reduce build size.

Use cases

  • Adding on-device dictation or transcription to a macOS or iOS app without a cloud API.
  • Building meeting note tools that label speakers in recordings.
  • Running always-on voice detection or wake-word style pipelines on Apple devices.
  • Adding local text-to-speech narration to iOS or macOS apps.

Things to know

  • Runs only on Apple platforms (macOS and iOS), since it depends on CoreML and the Apple Neural Engine.
  • Requires Swift 6.0 or newer.
  • Several TTS models, including Chatterbox, Paradee and LocalVQE, are marked as beta.

In collections

AI video & audio tools

Topics

#speech-to-text#text-to-speech#coreml#swift#speaker-diarization#voice-activity-detection#on-device-ai#apple-neural-engine#ane#asr

FAQ

What is FluidAudio?
A Swift SDK running speech models (ASR, TTS, VAD, speaker diarization) locally on Apple devices via CoreML and the Neural Engine. FluidAudio is a Swift SDK for running audio AI fully on-device on Apple platforms. Inference runs on the Apple Neural Engine through CoreML models, which the project says reduces memory use and CPU load compared with running on the CPU or GPU, making it suitable for background and always-on workloads.
Is FluidAudio free to use? What license does it have?
FluidAudio is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is FluidAudio still maintained?
Actively maintained. The last commit to FluidAudio was on Nov 2, 2026.
What can I use FluidAudio for?
Adding on-device dictation or transcription to a macOS or iOS app without a cloud API. Building meeting note tools that label speakers in recordings. Running always-on voice detection or wake-word style pipelines on Apple devices. Adding local text-to-speech narration to iOS or macOS apps.
Where can I get FluidAudio?
The source code and installation instructions are on GitHub: https://github.com/FluidInference/FluidAudio

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects