How to test a voice AI agent before production

Teams catch voice agent failures before callers do by testing in layers: text chat for conversation logic, AI-simulated callers for scenario coverage at scale, browser voice for the full speech pipeline, and real phone calls for telephony behaviour. Each layer costs more than the one below it and catches a class of failure the one below it cannot see.

What are the four layers of voice agent testing?

The four layers are text chat, simulated callers, browser voice, and real phone calls, ordered from the cheapest, fastest loop to the most realistic and most expensive. You test a voice AI agent by climbing this ladder: fix what each layer exposes before you pay for the next one.

1. 1. Text chatType messages to the agent. Exercises conversation logic, prompts, tools, and workflow branches. It does not exercise the speech stages.Cheapest loop, instant iteration
2. 2. Simulated callersAI personas play the customer: the angry caller, the rambler, the one who changes topic mid-sentence. Runs whole scenarios without human testers.Scenario coverage at scale
3. 3. Browser voiceTalk to the agent from the browser. Exercises speech-to-text, the language model, and text-to-speech as one live pipeline, with real latency and interruptions.Full speech pipeline, no phone line
4. 4. Real phone callsCall the agent on a real number through your connected telephony provider. The only layer that exercises number routing, transfers, and carrier audio.Full realism, carrier costs apply

Realism and cost rise with each step. The rule of thumb: never discover a problem on a paid phone call that a text chat would have caught.

All four layers exist in PeachDesk, the voice AI platform, so a team can run the whole ladder in one place instead of stitching together separate tools. For how the pipeline under test fits together, see voice agent architecture.

What should you evaluate at each testing layer?

Each layer has its own failure class: conversation logic and persona in text-based testing, turn-taking and speech behaviour in browser voice, and telephony behaviour in real calls. Evaluate for the failures that layer can actually produce.

Conversation logic and task completion

Script the calls your agent must handle: the happy path, the caller who gives information out of order, the caller who asks for something the agent cannot do. Check that the agent follows the workflow, calls the right tools with the right arguments, and reaches a completed task or a clean exit. Text chat and simulated callers carry this load; run the whole suite before you touch audio.

Persona drift

Persona drift is the agent slowly slipping out of role: getting casual where it should stay formal, inventing policies, or apologizing for things your business never promised. It shows up in long and adversarial conversations, which is exactly what simulated callers are for. Run extended personas that push, plead, and change subject, then read the transcripts for tone and factual consistency.

Interruption handling

Real callers barge in. On browser voice and phone calls, interrupt the agent mid-sentence and check three things: playback stops promptly, the pending reply is discarded instead of spoken over the caller, and the agent answers the interruption rather than resuming its old script. Interruption quality is an orchestration property, so it only appears once the speech stages are live.

Telephony behaviour

The final layer is the only one that touches the phone network. Verify that your inbound number routes to the right agent, that keypad navigation works where your flow uses it, and that transfers to a human or an external number complete with the outcome reported back into the conversation. Carriers meter these calls, which is why this layer comes last and runs a short, deliberate script.

Which metrics matter in voice agent evaluation?

Five metrics predict production behaviour: task completion, turn latency, interruption recovery, correct tool calls, and transfer correctness. Together they answer the only question that matters: did the call do what the caller needed, at a pace a real caller would tolerate?

Track these per scenario, not as one platform-wide average. An agent that completes the booking flow every time but stalls on the complaint flow has a complaint-flow problem, and only per-scenario numbers will show it.

How do you stop new changes from breaking a working agent?

Publish immutable versions and keep every call pinned to the exact version that served it. Then re-run your scenario suite against each new version before it takes traffic, so a prompt edit can never silently change live behaviour.

In PeachDesk, publishing creates an immutable agent version, and every call records which version handled it. That pin is what makes testing meaningful: the agent you tested on Tuesday is provably the agent that answered calls on Wednesday. When you change a prompt, a workflow branch, or a provider for any pipeline stage, you publish a new version, run the same ladder of tests against it, and only then route traffic to it.

The discipline that makes this work is a fixed scenario suite: a named set of text conversations, simulated-caller personas, and phone-call scripts that every candidate version must pass. Treat the suite the way you treat regression tests in software. When a real call surfaces a failure, add that call to the suite so the failure class stays covered forever.

Version history is an audit trail, not an undo button: it tells you exactly which version served which call, which is what you need to compare behaviour across versions.

What belongs on the pre-launch checklist?

The checklist is the ladder, run end to end against the exact version you intend to publish, with a named pass signal per layer. If any row fails, the agent is not ready, regardless of how good the demo sounded.

For the full product walkthrough behind these layers, see how PeachDesk works; unfamiliar terms are defined in the voice AI glossary.