Repolode
Submit
yizhi-chengzi

yizhi-chengzi/video-ai-talking

Local-first tool that turns a recorded talking-head clip and a script into a vertical MP4 with AI voice-over and automatic lip-sync.

View on GitHub
Stars
480
Forks
70
Open issues
0
Last commit
2 months ago

What is it?

video-ai-talking produces short talking-head videos from a single recording of a real person. You supply a front-facing close-up clip and the text to be spoken, or let an LLM draft the script from a topic. The tool generates voice-over with Volcano Engine or Alibaba Bailian CosyVoice, sends the clip and audio to Bailian VideoRetalk for lip-sync, and composites the result with FFmpeg into a vertical MP4 using a chosen subtitle and title style.

It runs entirely on your own machine. API keys are stored in the browser's localStorage, and tasks and finished videos are written to a local data folder. Video material can be inserted over specific subtitle lines to replace the picture for those moments. A mock mode lets you test the pipeline without calling external services.

Setup requires Node.js 20+, FFmpeg on the PATH, and the API keys for the lip-sync service plus at least one TTS provider. DeepSeek is optional for script writing.

Key features

  • 01Generates voice-over from text with Volcano Engine or Alibaba Bailian CosyVoice.
  • 02Lip-syncs a recorded clip to the audio via Bailian VideoRetalk.
  • 03Composites a vertical MP4 with FFmpeg and selectable subtitle styles.
  • 04Inserts local video material over chosen subtitle lines.
  • 05Optionally drafts scripts from a topic using DeepSeek.
  • 06Keeps API keys in the browser and task data on the local machine.

Use cases

  • Making short talking-head explainer videos without recording each take.
  • Producing vertical clips for short-video platforms from a single face recording.
  • Prototyping AI-dubbed talking-head content with mock mode before paying for API calls.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/yizhi-chengzi/video-ai-talking.git
  2. 2

    Enter the project folder

    cd video-ai-talking
  3. 3

    Copy the example env file

    cp .env.example .env
  4. 4

    Install dependencies and start the dev server, then open http://127.0.0.1:5175

    npm install && npm run dev

Things to know

  • Requires a Bailian VideoRetalk API key and FFmpeg installed locally.
  • Lip-sync and voice-over rely on paid cloud APIs (Alibaba Bailian, Volcengine, optionally DeepSeek).
  • Uploads reference video and voice audio to Alibaba Bailian temporary storage for processing.
  • Linux file picker needs Zenity installed.

Topics

#video-generation#lip-sync#text-to-speech#ffmpeg#talking-head#typescript#llm

FAQ

What is video-ai-talking?
Local-first tool that turns a recorded talking-head clip and a script into a vertical MP4 with AI voice-over and automatic lip-sync. video-ai-talking produces short talking-head videos from a single recording of a real person. You supply a front-facing close-up clip and the text to be spoken, or let an LLM draft the script from a topic. The tool generates voice-over with Volcano Engine or Alibaba Bailian CosyVoice, sends the clip and audio to Bailian VideoRetalk for lip-sync, and composites the result with FFmpeg into a vertical MP4 using a chosen subtitle and title style.
Is video-ai-talking free to use? What license does it have?
video-ai-talking is open source under the MIT license. Check the license terms before using it in commercial products.
Is video-ai-talking still maintained?
Updated within 6 months. The last commit to video-ai-talking was on Sep 3, 2026.
What can I use video-ai-talking for?
Making short talking-head explainer videos without recording each take. Producing vertical clips for short-video platforms from a single face recording. Prototyping AI-dubbed talking-head content with mock mode before paying for API calls.
How do I install video-ai-talking?
Clone the repository: git clone https://github.com/yizhi-chengzi/video-ai-talking.git Enter the project folder: cd video-ai-talking Copy the example env file: cp .env.example .env Install dependencies and start the dev server, then open http://127.0.0.1:5175: npm install && npm run dev
Where can I get video-ai-talking?
The source code and installation instructions are on GitHub: https://github.com/yizhi-chengzi/video-ai-talking

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects