模型與代理系統
Gander Open-Sources a Full-Duplex Multimodal Agent, Dividing Parallel Conversation and Long-Running Tasks Between a “Cerebellum” and “Cerebrum”
Gander brings streaming audiovisual conversation, interruption control, and asynchronous agent execution onto a single timeline, allowing the front-end model to continue interacting with users while the back-end agent works. The initial release includes Apache 2.0-licensed code and model weights, but the full dataset has yet to be released, and the official deployment configuration requires three GPUs.
Gander released its model weights, training code, and agent execution environment on September 9, aiming to resolve a common trade-off in voice agents: low-latency models are well suited to real-time conversation, while more capable reasoning agents need more time to plan and use tools. The system is therefore divided into a “cerebellum,” an orchestration and execution environment, and a replaceable “cerebrum.”
The cerebellum is based on MiniCPM-o 4.5 and arranges audio, video, text, and output into a causal sequence of one-second units. For each unit, it first predicts a `listen`, `speak`, `interrupt`, or `tool` control token, then generates up to eight text tokens. A separate Talker produces 50 S3 speech tokens per unit and streams output in roughly half-second chunks, allowing the model to continue receiving video and audio while synthesizing speech. The context retains up to 128 units.
For long-running tasks, the orchestration layer passes confirmed user instructions, visual input, and task state to a back-end agent; the initial release uses Codex as its built-in provider. While execution is underway, users can add constraints, cancel tasks, or respond to permission requests, while a generation fence prevents stale execution results from being delivered as the final answer. Compared with simply connecting speech recognition to a text-based agent, this adds a state machine for handling interruptions, task generations, and progress updates.
The official model was trained on 2.7 million mixed-data samples covering voice interaction, audiovisual understanding, the agent lifecycle, and interference examples. The report says it achieved 78.53% accuracy on Daily-Omni and correctly bound the final response to the appropriate task in all 45 delegation tests. However, there is currently no standardized benchmark for “full-duplex agents,” and several results still come from the team’s own evaluations. Engineering teams should also note that the full service configuration runs the Thinker, Talker, and ASR on separate GPUs; as of September 10, the dataset was still being released, leaving training reproducibility and cost comparisons incomplete.