Repolode
Submit
neural-maze

neural-maze/realtime-phone-agents-course

A week-by-week course for building an AI voice call center with FastRTC, Superlinked, Twilio, Opik and Runpod.

View on GitHub
Stars
1,109
Forks
253
Open issues
6
Last commit
9 months ago

What is it?

This is a multi-lesson course that walks through building a realtime AI voice agent system. The running example is a real estate company whose employees are voice agents that answer and place phone calls through Twilio, search property listings with Superlinked, and hold low-latency conversations powered by FastRTC.

The speech and model pieces cover speech-to-text with Moonshine and Faster Whisper, text-to-speech with Kokoro and Orpheus 3B, and deployment of open-source models on Runpod GPU pods. Later lessons add multiple avatars with distinct personas and voices, versioned prompts, and full tracing of each call through Opik.

Each lesson pairs a Substack article and live session with code pushed to its own branch and Jupyter notebooks. The course requires a working setup of API keys for Twilio, Runpod, Opik and Qdrant Cloud before the lessons can be run.

Key features

  • 01Inbound and outbound phone calls handled by voice agents via Twilio
  • 02Realtime speech-to-text and text-to-speech with local and hosted model options
  • 03Property search over structured and text data using Superlinked
  • 04Self-hosted Whisper and Orpheus models deployed on Runpod GPU pods
  • 05Multi-avatar agents with per-avatar personas, prompts and voices
  • 06Tracing of transcription, LLM, tool and TTS steps with Opik

Use cases

  • Learning how to build production-style voice agents for customer calls
  • Prototyping an AI receptionist or call center that searches a product or listing database
  • Comparing local and GPU-hosted speech models in a working voice pipeline

Quick start

  1. 1

    Read the setup guide in docs/GETTINGS_STARTED.md to configure your environment and install dependencies

  2. 2

    Start the call center application (FastAPI on port 8000 via Docker Compose)

    make start-call-center
  3. 3

    Expose your local server to the internet with ngrok

    make start-ngrok-tunnel

Things to know

  • This is a course rather than a plug-and-play tool, and it is built week by week.
  • Connecting real phone calls requires a Twilio account and an ngrok tunnel.
  • Several components rely on paid or GPU-backed services such as Runpod and Groq.

Topics

#voice-agent#realtime#speech-to-text#text-to-speech#twilio#vector-search#gpu#tutorial

FAQ

What is realtime-phone-agents-course?
A week-by-week course for building an AI voice call center with FastRTC, Superlinked, Twilio, Opik and Runpod. This is a multi-lesson course that walks through building a realtime AI voice agent system. The running example is a real estate company whose employees are voice agents that answer and place phone calls through Twilio, search property listings with Superlinked, and hold low-latency conversations powered by FastRTC.
Is realtime-phone-agents-course free to use? What license does it have?
realtime-phone-agents-course is open source under the MIT license. Check the license terms before using it in commercial products.
Is realtime-phone-agents-course still maintained?
No commits for 6+ months. The last commit to realtime-phone-agents-course was on Jan 10, 2026.
What can I use realtime-phone-agents-course for?
Learning how to build production-style voice agents for customer calls Prototyping an AI receptionist or call center that searches a product or listing database Comparing local and GPU-hosted speech models in a working voice pipeline
How do I install realtime-phone-agents-course?
Read the setup guide in docs/GETTINGS_STARTED.md to configure your environment and install dependencies Start the call center application (FastAPI on port 8000 via Docker Compose): make start-call-center Expose your local server to the internet with ngrok: make start-ngrok-tunnel
Where can I get realtime-phone-agents-course?
The source code and installation instructions are on GitHub: https://github.com/neural-maze/realtime-phone-agents-course

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects