多模態開發工具
VLM Run Gateway Unifies OCR, Video, and Vision Model APIs, but Model Names Still Do Not Guarantee Equivalent Inference
VLM Run has launched an OpenAI Chat Completions-compatible gateway that lets OCR, image captioning, video understanding, and specialized ViTs share endpoints and an MCP interface. It aims to expose differences in quantization and serving configurations, but current claims about performance, visual accuracy, and provider support rates are based primarily on the publisher’s own testing.
On September 4, VLM Run launched [Gateway](https://www.vlm.run/gateway), bringing open-weight OCR, general-purpose VLMs, detection, segmentation, pose estimation, and other models under a single OpenAI-compatible endpoint. Applications can continue using the Chat Completions SDK by changing only the `base_url`, model name, and image, video, or document input. The platform also provides an MCP server, allowing agents to invoke vision capabilities as tools. The public catalog currently lists 21 models along with their supported input types, context lengths, and pricing.
The service addresses a multimodal-routing problem often obscured by text-model APIs: the same model ID may actually map to different quantization formats, vLLM/SGLang parameters, or image preprocessing pipelines. Quantization errors that are invisible in text benchmarks can become significantly amplified in small-text OCR, table coordinates, or spatial relationships. The team therefore argues that the gateway should treat specific combinations of models and serving configurations as selectable execution targets, rather than assuming that endpoints sharing the same name are equivalent.
Video represents another compatibility gap. In a [Hugging Face technical overview](https://huggingface.co/blog/vlm-run/introducing-gateway), the team says that more than 80% of the popular routing providers it tested could not handle native video input, and even fewer allowed control over sampling FPS. Gateway, by contrast, lets customers specify the video model and processing method. For document pipelines, a unified response contract can also reduce the integration cost of A/B testing across general-purpose VLMs, specialized OCR systems, and smaller models.
For now, however, it is a hosted gateway—not an open-source runtime that lets users independently reproduce the entire routing and inference environment. Official figures claiming more than 3,000 output tokens per second do not include complete details about hardware, batch size, page resolution, or accuracy conditions. Likewise, “supporting the same API” does not mean that different models use consistent semantics for coordinates, Markdown, or tool outputs. Before adoption, engineering teams should retain source images and model-version metadata, build their own regression sets for vertical Chinese text, Traditional Chinese characters, tables, and low-resolution scans, and verify policies covering data retention, regional deployment, and failure retries.