AI 開發工具
Experiential Releases Open Agent Model Gateway, Automatically Routing Between Cost and Quality Based on Real-World Traces
Experiential unifies cloud, bring-your-own-key, and local models behind an OpenAI-compatible interface, with identity scoping, model allowlists, and hard budget controls. It can also build task routers from OpenTelemetry agent traces, although its cost-savings figures currently come primarily from the team’s own evaluations.
Experiential Labs released Experiential 0.7.2 on PyPI on August 28 and open-sourced the model gateway under the Apache 2.0 license. Developers can run a local endpoint with `pip install experiential`, allowing the OpenAI SDK to call hosted models, bring-your-own-key services, or local inference through the same `/v1/chat/completions` interface; the hosted version is also compatible with the Anthropic Messages API. The gateway can issue separate keys for different users, agents, and tools, configure model allowlists and daily or monthly budgets, and stop requests outright when a limit is reached.
The more technically distinctive feature is its ability to build routing policies from production traces. The system reads OpenTelemetry JSONL, replays representative tasks across different models, and then fits a router based on quality, latency, and price. The public coding-router uses a LoRA-fine-tuned Qwen3-Embedding-0.6B to compare new tasks against an evidence bank containing empirical results. If an input differs too greatly from the database, it is marked as out-of-distribution and falls back to the strongest model instead of risking the selection of a cheaper model.
On tests excluded from fitting, the team claims cost reductions of 83% for short interactions and LiveCodeBench, and 74% for long-running DeepSWE-like tasks, with quality differences remaining within statistical noise. However, these results have not yet been independently reproduced, and the evidence bank, model pricing, and provider behavior will continue to change. Engineering teams should also consider that anonymous PostHog telemetry is enabled by default, the expanded attack surface created by centralizing provider keys, and whether the OpenAI-compatible interface fully preserves the semantics of streaming, tool calls, error codes, and reasoning parameters.