Guide to Evaluating Live Voice Agents in ADK
How to test your voice-based AI agents before production deployment
1Challenges of Live Agents
Deploying voice AI agents requires more than just a good demo. They must take correct actions across spoken conversations where timing and recovery matter as much as content. Small prompt tweaks or model updates can silently break previously perfect behaviors. Tools may stop working, context can slip between turns, and interjections might get ignored.
2ADK's Native Live Evaluation
ADK now offers native live evaluation capabilities. You can test your voice agents with a simulated user that speaks turns as audio, score spoken responses, and do it all within the same evaluation loop used for text agents. This feature helps transition your agents from 'works in demo' to 'measured and trusted' without leaving ADK.
3Example Workflow: Three-Stage Voice Agent
Our example uses a graph-based workflow with three single-purpose live agents. The greeter_agent verifies caller identity. The dob_verifier_agent requests birth date and calls a validation tool. The goals_agent shares appointment details. All stages run on gemini-live-2.5-flash-native-audio model, maintaining context across handoffs.
4User Simulation and Scenarios
Define test cases in JSON format. With 'conversation_scenario', you specify goals and personas while the user simulator improvises turns. For example, the 'NOVICE' persona makes the simulated user share only high-level goals, testing how well the agent drives conversation. Scenarios automatically end when their plan is satisfied, with max_allowed_invocations preventing runaway conversations.
5Fixed Conversation Tests
As an alternative to dynamic scenarios, you can script every user turn verbatim. The 'conversation' array lets you define each user utterance as text. These static test cases serve as equally valid inputs for live runs as simulated users, providing precise control over test conditions.
6Speech Synthesis and Live Streaming
Configure your tests by adding 'live_model_config' to point ADK at the 'llm_audio' user simulator. Each user turn gets synthesized to speech using your chosen Gemini TTS voice and streamed to the live agent. Parameters like timeout_seconds let you customize the testing environment.
7Comprehensive Evaluation Criteria
Define assessment metrics like 'rubric_based_multi_turn_trajectory_quality_v1'. For instance, the 'verifies_identity_first' rubric requires agents to confirm caller identity before sharing appointment details. You can fine-tune measurements with threshold values and judge model options.
8Pre-Production Safety Net
ADK's live evaluation feature creates a robust safety net for voice AI agents. By simulating real user interactions, it lets you test production scenarios in advance. This approach enables development of innovative voice applications without compromising user experience.
Original article: Google Developers Blog — ADK'de Canlı Ses Ajanlarını Değerlendirme Rehberi