AI 研究
Shared Inference Cost Study Publishes H100 Results, Showing How Metering Changes Tenants’ Cost Shares
unalloc consolidates cluster and model bills. The study found that different metering methods shifted the same tenant’s share of costs by roughly 12–14 percentage points. Hardware validation consisted of one short run per workload, so the results do not establish a definitive allocation.
Independent researcher Timothy Urista submitted a study on September 21 examining how multitenant inference services allocate GPU and model API spending. Accompanied by the open-source tool unalloc, the paper brings Kubernetes resource costs, model gateway logs, and provider bills into a single ledger. Its central finding: even when the totals reconcile, cost attribution can change substantially depending on metering rules and label handling. [Paper abstract](https://arxiv.org/abs/2609.24991)
The tool converts sources such as OpenCost and LiteLLM into a common `CostRow` format, represents monetary amounts using decimal values, and normalizes team, project, and cost center fields across systems. Alongside amounts that cannot be attributed, reports separately flag spending allocated through fallback fields, preventing an arbitrary name match from being mistaken for identification of the responsible team. [Public repository](https://github.com/timurista/unalloc)
The study’s hardware validation used a single H100 80GB GPU, vLLM 0.29.0, and Qwen2.5-7B-Instruct in BF16, with prefix caching enabled, to replay synthetic multitenant traffic. Each of four workloads ran for two minutes. When costs were allocated by token count, the retrieval-oriented tenant accounted for 16.5%–18.9% of total costs. When time was divided equally among outstanding requests every 50 milliseconds, its share was 4.7%–5.3%—a difference of roughly 12–14 percentage points. [Experimental methods and results](https://arxiv.org/pdf/2609.24991)
These differences hinge on how cost is defined. OpenCost’s technical documentation from July of this year already distinguished two measures: allocated cost includes reserved resources and shared infrastructure that keep a model available, while usage cost counts only actual inference consumption and accounts for cache hits. The gap can reflect idle capacity needed to maintain low latency, so deployment efficiency cannot be judged solely by a per-token price. [OpenCost cost model](https://opencost.io/blog/opencost-llmd-inference-cost/)
Ledger consolidation introduces a separate pitfall. The repository warns that ingesting both gateway and provider spending can count the same model call twice, making an explicit choice of source necessary. The tool provides invoice reconciliation, a list of missing labels ranked by monetary amount, and thresholds for the unattributed share of costs that can be integrated into continuous integration pipelines. These mechanisms turn allocation rules into an engineering process that can be checked. [Tool usage and data paths](https://github.com/timurista/unalloc)
Neither allocation algorithm offers a uniquely correct measure of cost, and equal time sharing is itself an approximation. The hardware evidence is limited to one model, one GPU, and one short run per workload, with no uncertainty estimates from repeated runs. Engineering teams therefore still need to rerun the tests using their own prompt lengths, cache hit patterns, and multiturn traffic before deciding on an internal cost allocation policy. [Paper limitations and reproducibility notes](https://arxiv.org/pdf/2609.24991)