GitHub Repo
vLLM Community Proposes MoE LoRA Backend Fix; Explicit Triton Selection May Still Fail to Start on Blackwell
Reports from the vLLM community on October 12–13 say MoE LoRA backend selection may ignore an explicit setting and trigger a startup error in fully sharded mode. The proposed fix adds configuration checks and unit tests, but has not been merged and lacks full GPU validation.
On October 12, the vLLM community reported a backend selection bug affecting unquantized MoE LoRA: under certain conditions on Blackwell GPUs, the system may still select FlashInfer’s TRT-LLM path even when --moe-backend triton is specified. If --fully-sharded-loras is also enabled, the service triggers an assertion during startup profiling. The reported environment used v0.31.0, an SM103 GPU, CUDA 13, and BF16 with Qwen3.5-35B-A3B. The reporter also noted that the same logic exists in the development branch. Issue report
The report traces the cause to the backend selection function: the LoRA branch checks TRT-LLM support conditions before handling the backend explicitly selected by the user. The selected implementation then calls the LoRA operation with add_inputs=False, but the lower-level kernel does not support combining this parameter with fully sharded mode. As a result, simply adding the Triton option does not avoid the error; the author says that after locally changing the selection condition, the same command started successfully and served requests. Reproduction and code path
This gap raises the question of whether deployment settings actually take effect. The official documentation defines --moe-backend as the interface for selecting the MoE expert computation kernel. Fully sharded LoRA shards the complete LoRA computation across the tensor-parallel configuration; the documentation says it may benefit long sequences, high ranks, or larger parallel sizes. For engineering teams, this suggests that when tuning performance parameters, they should also verify the active backend in the startup logs. Engine arguments documentation
PR #61301, proposed on October 13, aims to honor an explicitly selected Triton backend and pass the fully sharded setting into the MoE compatibility check. With automatic selection, the system can fall back to Triton; an explicitly requested, incompatible TRT-LLM combination instead raises a configuration error early. The author listed three passing unit tests, but the test record also shows that two tests were skipped because of CUDA/ROCm markers on a CPU host, so this does not constitute full GPU validation. Proposed fix
As of this review, the proposal remains open and is marked as needing merge conflict resolution. Follow-up should track maintainer review, testing on actual Blackwell hardware, and inclusion in an official release. Current evidence is insufficient to conclude that all MoE LoRA deployments will fail. Proposal status