Opening soon
Voice Agents & Conversational AI
Learn how voice agents recognize speech, work out intent, speak back and manage a conversation.
5 chapters · 20 lessons
Want a note when it opens?
Join the waitlist and we'll email you when the course is ready.
What you'll cover
The chapters
The plan for the full course. Lessons open when the course does.
- 01How voice agents differ from text chatbots4 lessons
- 02Speech recognition and transcription4 lessons
- 03Understanding intent in spoken language4 lessons
- 04Text-to-speech and voice design4 lessons
- 05Managing dialogue in a voice agent4 lessons
Self-check
How much do you already know?
5 quick questions. Pick an answer to see why it's right or wrong. Nothing is saved.
Question 1 of 5
What is the data throughput of a standard voice agent audio stream at 16kHz, 16-bit, mono?
Question 2 of 5
A voice agent is deployed on a support line where callers often interrupt it to correct it. Which orchestration responsibility is most directly relevant to handling this well?
Question 3 of 5
A voice agent is deployed for a multilingual customer service line where callers frequently switch between English and Spanish mid-sentence. Which ASR challenge does this represent?
Question 4 of 5
A team is evaluating whether to use streaming ASR partials to pre-warm their LLM context window. What is the key engineering cost they must account for?
Question 5 of 5
A healthcare voice agent is deployed in a noisy hospital environment. WER increases from 4% to 20%. What is the most significant downstream risk?
In the news
Recent stories on this topic
From NewsPals, our sister newsroom.
- Artificial Analysis Speech Agent Arena puts Gemini 3.1 Flash Live Preview - Minimal on top (opens in new tab)
The blind voice benchmark lists Google first by preference, with 768 samples and a 74.6% task success rate.
- Fish Audio’s $52M Seed Shows One AI Voice Model Serving Two Markets (opens in new tab)
The voice startup is selling creator tooling and enterprise infrastructure from the same core model layer.
Key terms
Words you'll meet in this course
- Automatic speech recognition (ASR)
- Software that converts spoken audio into text, the first step in most voice agents.
- Word error rate (WER)
- The share of words a speech recognizer gets wrong, counting substitutions, deletions and insertions against a reference transcript.
- Slot filling
- Collecting the specific details a request needs, such as a date or a name, by extracting them from speech or asking for what is missing.
- Text-to-speech (TTS)
- Software that turns written text into spoken audio, giving a voice agent its voice.
- Prosody
- The rhythm, stress, pitch and pacing of speech, which carry meaning and emotion beyond the words themselves.
- Barge-in
- A caller speaking while the agent is still talking; a well-built agent detects it, stops its own audio and listens.
- Endpointing
- Deciding when a speaker has finished a turn, using pauses and other cues, so the agent knows when to respond.
- Dialogue state
- What a voice agent tracks about the conversation so far, such as the user's goal and the details already collected.
Help us choose
Which course should open next?
Pick the one you'd take first. Your vote helps us decide what to finish next.