A production voice agent has three functional parts: ears, a brain, and a voice. The ears are speech-to-text, converting caller audio to text in real time. The brain is a language model, deciding what to say next and calling tools or knowledge bases when it needs facts or actions. The voice is text-to-speech, synthesizing the reply. Around these sits the orchestration layer: the call flow, the telephony or WebRTC connection, and the platform services that make the agent operable.
What are the components of a voice agent architecture?
Five components cooperate on every call. Four of them sit on the audio path; the fifth coordinates everything.
- Caller audioTelephony or WebRTC transport carries sound in both directions
- Speech-to-textThe ears: streaming transcription of what the caller says
- Language modelThe brain: decides the next utterance, calls tools and knowledge bases
- Text-to-speechThe voice: synthesizes the reply as natural audio
- Caller hears the replyThe loop repeats until the call ends, in fractions of a second per turn
The orchestration layer wraps this loop: the workflow that defines the conversation, tool and knowledge-base integrations, call control such as transfers, and the testing, analytics, and cost tracking that make the agent operable in production. Without orchestration you have a demo, not a system.
What is the difference between cascaded pipelines and speech-to-speech models?
In a cascaded architecture, speech-to-text, the language model, and text-to-speech are separate components chained together. You can swap providers per stage, inspect the transcript at every step, and route each stage to a different deployment, cloud or local. In a speech-to-speech architecture, one model consumes audio and emits audio directly, which can reduce latency and preserve tone, but removes per-stage choice and transcript visibility. Cascaded architectures are what make per-stage mix and match, and therefore cost and data control, possible.
What does the orchestration layer actually do?
Orchestration turns a model loop into a business process. In PeachDesk it is the visual workflow: a canvas of 16 core node types connected by conditional edges, with deterministic equation transitions where a call must follow exact rules. The workflow decides what the agent says, which tools it can call, when it looks up a knowledge base, when it transfers to a human, and how the call ends.
Around the workflow, the platform supplies the operating surface: testing by text chat, simulated callers, browser voice, or real phone calls; immutable published versions so every call is pinned to the exact agent version that served it; and per-call analytics and cost records so operators can see what each conversation did and cost.
How does PeachDesk implement this architecture?
PeachDesk is a voice AI platform built on the cascaded model: a real-time voice pipeline with pluggable providers per stage, driven by the visual workflow builder. Each pipeline stage independently routes to Frontier mode (metered cloud providers), your own keys (BYOK), or Local mode (a full open-model stack on your own GPUs), so cost and data decisions are made per stage, per agent. For the product walkthrough, see how PeachDesk works; for term definitions, the voice AI glossary.
Build on this architecture
The documentation covers workflows, providers, telephony, and the API surface in implementation detail.