推論系統／MoE
ACE Skips Low-Contribution MoE Experts Per Token, Accelerating Prefill by Up to 2.25× on A100
ACE estimates expert contributions from SwiGLU weights and router directions without retraining the model or using calibration data. The authors achieved better quality at high skip rates across three MoE models, though the performance results remain limited to a single A100 and thresholds matched to the workload.
Although MoE models activate only a subset of experts for each token, conventional fixed top-k routing still requires every token to execute the same number of expert slots. A new study, ACE, aims to eliminate another layer of redundancy. It preserves the original checkpoint and router, requires no separately trained configurator, and determines at inference time which selected secondary experts can be skipped for each token. The highest-weight top-1 expert is always retained.
ACE uses two offline signals. Global Spectral Proxy (GSP) jointly analyzes each SwiGLU expert’s gate, up, and down projections together with the preceding RMSNorm to estimate its global transformation capacity. Router-Conditioned Refinement (RCR) centers the router weights in the same layer to construct each expert’s preferred input direction, then measures response strength along that direction. The online stage requires only table lookups, combining the results with the current token’s gate value, and a small amount of scalar computation. The system skips an expert only when both GSP and RCR judge its contribution to be low.
The authors evaluated Qwen3-30B-A3B-Instruct-2507, Qwen3.6-35B-A3B, and Gemma-4-26B-A4B-it on WikiText-2 and seven tasks, including ARC, PIQA, MATH-500, GPQA-Diamond, HumanEval, and LiveCodeBench. At a 50% expert skip rate on Qwen3.6-35B-A3B, for example, ACE reduced WikiText-2 perplexity by 7.96% and increased average accuracy across the seven tasks by 4.15 percentage points compared with the strongest baseline. This is a comparison under the same skip budget, not evidence that ACE comprehensively outperforms the full BF16 model.
With optimized expert dispatch on a single NVIDIA A100 at a 60% skip rate, ACE improved time to first token by 1.72× to 2.25× and reduced per-token latency by 1.31× to 1.41×. With an input length of 1,024 and batch size 1, TTFT fell from 270.9 ms to 120.3 ms. The public repository includes an end-to-end pipeline and tests, but omits some auxiliary experiments, while the README’s calibration procedure still uses WikiText tokens to map target skip rates. The paper acknowledges that its results come from single deterministic runs using workload-matched thresholds. Future work must evaluate cross-domain threshold transfer, multi-GPU expert-parallel communication, and integration gains in production runtimes such as vLLM and SGLang.