醫療 AI
AMIE Video Splits Real-Time Consultations Across Three Agents, Scores 83% in Simulated Clinical Evaluation
Google separates low-latency conversation, in-depth clinical planning, and audiovisual perception into three asynchronous agents. Its overall score across 100 simulated video consultation scenarios exceeded the physician group’s 68%. However, the study used only trained patient actors, while the models, code, and audiovisual data have not been released, so the results cannot be directly extrapolated to real-world clinical deployment.
Google Research and Google DeepMind unveiled AMIE Video on August 10, transforming the previously text-oriented medical dialogue system into a real-time audiovisual architecture. Rather than requiring a single model to balance response speed with comprehensive reasoning, the system divides the work among three asynchronous agents. Talker uses Project Astra infrastructure and Gemini 3 Flash, reads only the most recent five seconds of video, and generates low-latency spoken responses. Planner uses Gemini 3.1 Pro to maintain a symptom summary, differential diagnosis, management plan, and conversation milestones, performing in-depth reasoning as frequently as once every ten seconds. Perception tracks cues such as coughing, posture, speaking rate, and movement over a longer audiovisual window, writes them to persistent observation memory, and makes them available to Talker. The [paper](https://arxiv.org/abs/2608.09861) also introduces single-turn “capability unit tests” and multi-turn simulated evaluations intended to assess perception, reasoning, and interaction failures separately.
The study used 100 clinical scenarios that actors could portray to compare AMIE Video, AMIE Text, and ten U.S. primary care physicians. Twenty additional physicians served as evaluators, while fifteen trained patient actors assessed the user experience. AMIE Video scored 83% on case-specific rubrics, compared with 68% for the physician group. Its top-choice differential diagnosis accuracy was 91% versus 77%, while its score for guiding physical examinations over video was 72% versus 39%. However, the difference in top-3 diagnostic accuracy was no longer statistically significant, and patient actors still preferred human physicians for rapport and the sense of collaboration.
From an engineering perspective, the architecture turns the common real-time-agent conflict between “fast responses” and “slow reasoning” into a state-synchronization problem. It also introduces new failure modes, however, including stale Planner information and cases in which perception is correct but fails to trigger an action. The system still relies on discrete turn-taking and cannot overlap speech or interject immediately in response to visual cues as a person can. High-frequency tremors, subtle emotions, and precise anatomical localization also remain weaknesses. Google’s earlier [real-world clinical feasibility study](https://research.google/blog/exploring-the-feasibility-of-conversational-diagnostic-ai-in-a-real-world-clinical-study/) still used a physician-supervised, text-based history-taking workflow. The next steps should therefore focus on real patients, calibration across demographic groups, latency, and consistency of agent state—not merely overall OSCE scores.