代理框架
Pydantic AI 2.40 Adds Playback-Calibrated Barge-In Interruptions for Realtime Voice Agents
The new release lets applications interrupt responses when users speak over them based on the number of audio bytes actually played, while keeping conversation state in sync. It also adds an out-of-band prompt queue and an event-listening interface, so realtime agents no longer need to disguise every control signal as a user turn.
Pydantic AI 2.40.0 focuses on one of the hardest state-management problems to conceal in realtime voice agents: the amount of audio generated by the model is not the same as the amount the user actually heard. Sessions can now enable `handle_barge_in=True` and report the number of bytes already played by the audio player through `interrupt(played_bytes=...)`; `played_audio_bytes` lets applications track that progress. When a user speaks over a response, the framework can therefore cancel subsequent output and keep conversation history aligned more closely with the portion that was actually played, preventing the model from assuming in the next turn that the user heard the entire response.
The control plane also gains several composable interfaces. Programs can use `RealtimeSession.enqueue()` to insert out-of-band prompts without presenting monitoring, policy, or backend events as human messages; `send(..., respond=)` explicitly controls whether the model should be asked to respond after text is sent. `provider_factory` defers realtime model resolution so the provider can be selected at runtime, while `@agent.on_event` provides a unified event-listening hook suitable for audit logging, telemetry, or custom state machines.
This update also fixes several edge cases that could cause voice sessions to hang or corrupt conversation history: race conditions when canceling during barge-in, rollbacks of user turns caused by non-`bytes` audio, audio and transcript streams failing to terminate after tool failures, and the erroneous transmission of the `reasoning` parameter to older versions of `gpt-realtime`. Engineers must still manage player buffering, sample formats, and provider-specific cancellation semantics themselves; converting byte counts into “how much the user heard” is reliable only when the encoding and playback pipeline are consistent. In addition, Pydantic AI releases frequently, so production deployments should pin the version and include end-to-end tests for barge-in, tool failures, and network jitter.