推論系統
Yutori Reveals Browser Agent Inference Architecture: 100K-Token Inputs, Short Tool Outputs, and Prefix Caching to Reduce Per-Step Costs
Yutori’s Navigator calls a vision-language model dozens of times during each browser task, with historical context growing to roughly 100K tokens. Together AI handles this workload using vLLM prefix caching and separate latency targets, although the claimed 2× speedup and 4–5× lower cost remain vendor-reported comparison results.
Yutori and Together AI have disclosed the production inference design behind the Navigator browser agent, revealing a workload profile markedly different from that of conventional chat services. At every step, the agent captures the browser screen, appends the new screenshot and complete action history to the context, and asks the model to produce a mouse, keyboard, or JavaScript tool action. A single task can run for 10 to 15 minutes and generate dozens of model calls. Inputs grow from a few thousand to roughly 100K tokens, while outputs are usually short, making repeated prefill—not long-form decoding—the primary bottleneck.
[Together AI’s technical case study](https://www.together.ai/customers/yutori) says its service is built on vLLM with prefix caching enabled. Most of the conversation history is identical between consecutive steps, so the previously computed KV cache can theoretically be reused directly, with only the latest screenshot and action result processed again. If the scheduler is unaware of cache locality, moving requests between GPUs can eliminate cache hits. Per-step latency for browser agents therefore cannot be assessed solely by average tokens per second.
The platform also divides the workload into two categories: background Scouts prioritize batch throughput and per-run cost, while interactive agents whose users are waiting require strict tail-latency controls to prevent browser sessions from timing out. Together claims that Navigator is 2× faster per step and costs 4–5× less to run than comparable frontier solutions, while offering a 99.9% SLA. However, it has not disclosed the GPU models, batching strategy, cache-hit rate, or complete cost benchmark, so the results cannot yet be reproduced independently.
On the model side, Yutori says Navigator is trained through supervised fine-tuning and reinforcement learning using both synthetic data and data from real websites. [Official information about Navigator n1.5](https://yutori.com/blog/introducing-n1-5) reports a 97.3% success rate in human evaluations on Online-Mind2Web. However, websites change continuously, and online benchmarks are also sensitive to login state, geographic region, and page version. The engineering team’s next priorities should include cache isolation, screenshot-data retention, protection against cross-tenant KV cache leakage, and real-world task failure rates following website redesigns—not merely the average speed figures published by the vendor.