AI 基礎設施
LiteLLM 1.100.0 Brings Shared Budgets, MCP Token Validation, and Vertex Interactions into a Single Gateway
The new release makes shared budgets for model access groups persistent across billing cycles and adds RS256 signatures and RFC 7662 introspection for MCP sessions. It also expands routing for Vertex AI, Bing grounding, and speech models, but removes an existing token calculation function, so direct imports and database migrations must be checked before upgrading.
LiteLLM released version 1.100.0 on September 6. The update goes beyond adding model names, advancing identity, budgeting, and provider routing together for multi-tenant AI gateways. The new version supports shared budgets for “model access groups,” rather than limiting quotas to individual virtual keys. A new `LiteLLM_BudgetWindowSpend` table has been added to the backend, with the spend writer maintaining spending for each billing cycle and the read path now using this aggregate table. This is more practical for deployments in which one team uses multiple keys to access the same set of models, but upgraders should first validate Prisma migrations, billing-cycle boundaries, and concurrent spend-deduction behavior.
For the MCP gateway, session tokens can now use asymmetric RS256 signatures, with RFC 7662 token introspection allowing external services to check token status without sharing a symmetric secret. The system can also bulk-import Anthropic MCP connectors through the API or admin interface. At the provider layer, the release adds native support for the Vertex AI Interactions API, Bing Search grounding, and Gemini 3.5 Transcribe through Vertex’s `/v1/audio/transcriptions` endpoint. It also includes multiple compatibility fixes for the Responses bridge, Anthropic thinking blocks, video, streaming, and billing.
The security and operational details are equally noteworthy. The release fixes issues that could cause Vertex passthrough to forward a caller’s virtual key, retry logs to retain forwarded credentials, and RAG ingestion to apply upload controls incompletely. Official container images also include a verification method using cosign signatures. However, this gateway release has a broad change surface, and the maintainers have not provided end-to-end performance or budget-consistency benchmarks. `prompt_token_calculator` has also been removed in a breaking change, so applications that imported it directly from `utils` will fail. Before production deployment, teams should use traffic replay to test Responses, MCP, billing, and fallback paths instead of merely confirming that the service starts.