AI 基礎設施
Hetzner Trials an OpenAI-Compatible Inference API to Test European Hosting Demand With a Single FP8 MoE Model
Hetzner has quietly opened an experimental LLM inference service that currently offers only Qwen3.6-35B-A3B-FP8. Existing OpenAI SDKs can connect by changing the base URL. Low-load community tests recorded a median time to first token of 153 milliseconds and roughly 224 tokens/s, but the service has no SLA, billing, or production guarantees.
German cloud and hosting provider Hetzner has begun testing its own LLM inference API. Available through Hetzner Experiments, the service exposes an interface compatible with OpenAI Chat Completions. Developers can continue using the OpenAI Python SDK by changing `base_url` to `https://inference.hetzner.com/api/v1` and supplying a token generated by the experimental platform. This lowers the integration cost of switching existing applications to another inference provider.
The model catalog currently contains only `Qwen/Qwen3.6-35B-A3B-FP8`. It is a Mixture-of-Experts (MoE) model with approximately 35B total parameters and around 3B parameters activated per token. It supports text, images, and a 262K context window, with weights stored in FP8. This configuration can deliver high throughput with relatively limited VRAM, making it suitable for summarization, classification, extraction, and pre- and post-processing for retrieval-augmented generation (RAG). It should not, however, be treated as a replacement for a frontier reasoning model.
On July 23, a Sliplane engineer tested the service with a single client. Across seven short requests, the median time to first token (TTFT) was 153 milliseconds; five longer generations reached approximately 224 output tokens per second. The tests also found that format adherence, information retrieval, and image input were broadly functional, although the model failed two simple arithmetic questions. Because these results include no concurrent load, tail-latency, or sustained-load data, they should be treated only as early observations.
The infrastructure direction is more technically significant. An OpenAI-compatible interface lets Hetzner package shared GPU capacity as a request-based service without requiring customers to operate vLLM themselves. For teams that already deploy applications in European hosting environments, it could also reduce cross-provider networking and governance costs. Hetzner currently labels the service explicitly as an experiment, however: it is free and has no billing system, SLA, or production commitment, while neither its support scope nor underlying hardware has been disclosed.
The next points to watch are the model catalog, pricing, rate limits, batch and streaming semantics, and whether Hetzner adds a Data Processing Agreement (DPA), a data-retention policy, and availability commitments. Until those conditions are in place, engineers should use only synthetic or non-sensitive data for compatibility and load testing. They should not assume that the service meets formal data-governance requirements merely because its servers are located in Europe.