Repolode
Submit
amywork777

amywork777/lipflow

Hold a key, silently mouth words on your webcam, and Lipflow types the recognized text at your cursor, running locally.

Open-source alternative to:Wispr Flow

View on GitHub
Stars
633
Forks
73
Open issues
11
Last commit
13 days ago

What is it?

Lipflow is a lip-reading dictation tool. You hold a key, mouth what you want to say without making a sound, and release it. The recognized text is pasted at the cursor in whatever app is active, using only a webcam and no microphone for the default mode.

The lip reader is based on Auto-AVSR visual-only speech recognition, with MediaPipe face landmarks used to crop mouth regions from each frame. An optional LLM cleanup step corrects likely errors using Claude, the Codex CLI, Ollama, an on-device MLX model, or offline rules. Setup can import Wispr Flow dictation history locally and fine-tune the models on the user's own face and phrasing on Apple Silicon.

The main build targets macOS on Apple Silicon, with Windows and Linux support also described. An optional whisper mode adds the microphone for better accuracy.

Key features

  • 01Silent lip-reading input with only a webcam, no microphone needed by default.
  • 02Pastes recognized text at the cursor in any app.
  • 03Runs locally, with an optional on-device MLX model for LLM cleanup.
  • 04Optional LLM cleanup via Claude, Codex CLI, or Ollama to fix misread words.
  • 05Personal training on your own face and phrasing, with held-out evaluation.
  • 06Custom word list that biases recognition toward names and terms.

Use cases

  • Dictating messages or notes in a quiet office or shared space without speaking aloud.
  • Typing with a webcam when you cannot or prefer not to vocalize.
  • Experimenting with visual speech recognition on your own recordings.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/amywork777/lipflow.git ~/code/lipflow
  2. 2

    Run the setup script (installs dependencies, models, and builds the app on macOS)

    cd ~/code/lipflow && ./setup.sh
  3. 3

    Open the Lipflow app and complete first-launch setup

    open /Applications/Lipflow.app

Things to know

  • Setup requires an Apple Silicon Mac on macOS 13 or later for the main path; Windows and Linux are also supported with differences.
  • Silently mouthed speech is less accurate than filmed speech, so expect more errors on your own webcam.
  • The LRS3-trained model weights are for non-commercial research use.

Topics

#lip-reading#speech-recognition#visual-speech#dictation#macos#computer-vision#pytorch#mediapipe

FAQ

What is lipflow?
Hold a key, silently mouth words on your webcam, and Lipflow types the recognized text at your cursor, running locally. Lipflow is a lip-reading dictation tool. You hold a key, mouth what you want to say without making a sound, and release it. The recognized text is pasted at the cursor in whatever app is active, using only a webcam and no microphone for the default mode.
Is lipflow free to use? What license does it have?
lipflow is open source under the MIT license. Check the license terms before using it in commercial products.
Is lipflow still maintained?
Actively maintained. The last commit to lipflow was on Oct 7, 2026.
What can I use lipflow for?
Dictating messages or notes in a quiet office or shared space without speaking aloud. Typing with a webcam when you cannot or prefer not to vocalize. Experimenting with visual speech recognition on your own recordings.
How do I install lipflow?
Clone the repository: git clone https://github.com/amywork777/lipflow.git ~/code/lipflow Run the setup script (installs dependencies, models, and builds the app on macOS): cd ~/code/lipflow && ./setup.sh Open the Lipflow app and complete first-launch setup: open /Applications/Lipflow.app
Where can I get lipflow?
The source code and installation instructions are on GitHub: https://github.com/amywork777/lipflow

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects