開放模型與推論
openPangu 2.0 Pro Releases Open 505B MoE Weights, Supports a 512K Context with 18B Active Parameters
Huawei’s openPangu team has released a 505B mixture-of-experts model trained on Ascend NPUs, activating approximately 18B parameters per token. The model introduces hierarchical sparse attention and three-head multi-token prediction, though its performance figures currently come primarily from the team’s own evaluations.
Huawei’s openPangu team has released the openPangu 2.0 Pro weights and inference code, offering both thinking and non-thinking modes. The model has approximately 505B total parameters, activates about 18B per token, and was pretrained on roughly 34T tokens. It is listed as supporting contexts of up to 512K tokens. The technical focus of this release is not merely the model’s scale, but an effort to keep an ultra-large MoE deployable on Ascend hardware.
The attention layers interleave DSA and sliding-window attention at a 1:2 ratio: SWA handles local relationships, while DSA performs sparse global aggregation, avoiding the cost of full attention at every layer for long sequences. The network also replaces conventional single-path residual connections with four-stream mHC and adds three MTP prediction heads that propose the next three tokens at once for verification through self-speculative decoding. The official inference path uses omni-infer rather than an off-the-shelf vLLM endpoint. The Hugging Face page also currently lists no third-party Inference Provider, so downloading the weights does not mean they can be used directly with existing CUDA serving stacks.
The team’s self-reported evaluations in thinking mode include 68.5 on SWE-bench Verified, 85.7 on LiveCodeBench V6, 81.1 on TAU2-Bench, and 87.9 on GPQA-Diamond. These results use different aggregation methods, including Avg@3 and Avg@4, and therefore cannot be compared directly with models reporting only single-run success rates. Test prompts, tool environments, and inference budgets can also significantly affect agent benchmark results. The model is distributed under the dedicated OpenPangu Model License 2.0; downloadable weights alone should not be taken to mean that it is open source under a standard Apache or MIT license.
Engineering teams should next watch two things: omni-infer’s KV cache requirements, time to first token, and multi-user throughput with real 512K-token inputs; and whether the community can reproduce the official results on non-Ascend hardware. Only once conversion tools and independent evaluations become available is this model likely to move beyond a hardware-ecosystem showcase and become a portable long-context foundation model.