推論系統
CoBa Dynamically Allocates Inference Compute, Matching Best-of-16 With 58.9% Fewer Weighted Tokens
CoBa initially generates a small number of candidate answers, then uses disagreement and verification scores to decide whether to sample additional candidates or activate a stronger verifier. It achieved 85.13% macro-average accuracy across five reasoning benchmarks, although the results came from offline replay of pre-generated candidate pools rather than testing in a live service.
Test-time compute for reasoning models typically follows a few fixed recipes: think for longer, generate more answers at once, or have another model score each answer. CoBa reframes all three as a single budget-allocation problem, allowing a router at each step to choose whether to continue sampling, use a lightweight or strong verifier, or stop immediately.
The process begins with Qwen3-14B, Phi-4-reasoning, or Qwen3-8B generating two candidates. It assesses stability using answer agreement and scores from Qwen3-8B. When uncertainty remains, the pool expands to as many as eight candidates, after which the four highest-ranked candidates are passed to Qwen3-14B for in-depth verification. The final score combines answer frequency, lightweight judgments, process scores when available, and strong-verifier results. The study also calculates cost as “model parameters in billions × input and output tokens,” approximating the compute difference when models of different sizes process the same number of tokens.
Across 1,043 problems and 3,129 problem–generator combinations from MATH-500, AIME 2024/2025, AMC 2023, and the difficult subset of Reasoning Gym, CoBa-Routed-Strong achieved 85.13% macro-average accuracy, roughly matching the 85.12% of Best-of-16 majority voting while using 58.9% fewer weighted tokens. Compared with self-evaluation-weighted voting, which reached 85.20%, it reduced cost by 49.1%. On AIME 2024, accuracy rose from 65.6% with a single sample to 82.2%. On AIME 2025, however, it remained well below the candidate-pool oracle upper bound of 91.36%, indicating that some errors occurred because no correct answer was generated at all, rather than because the router made the wrong choice.
The main limitation is that every method was evaluated through offline replay over pre-generated pools of 16 candidates. The paper implements proxy versions of some recent methods and does not release a complete codebase. Parameter-weighted token count is not equivalent to actual GPU time or cost, and comparisons across architectures are especially coarse. The next engineering step is to determine whether online, incremental generation can preserve this accuracy–cost curve after accounting for routing overhead, batching efficiency, and tail latency.