推論系統
llama.cpp Mainline Adds Maple 20B-A1B Ternary MoE, With Initial Support Targeting CPUs
Architecture support merged on September 14 allows Maple-Preview to be loaded and converted without relying on DeepGrove’s llama.cpp fork. The model activates only about 1 billion parameters per token, but GPU backends, quality evaluations, and official speed claims still require separate verification.
llama.cpp merged support for Maple-Preview on September 14, adding GGUF constants, a Hugging Face converter, model architecture registration, and tests. Maple is a 20B-A1B reasoning model with 24 layers and 256 experts, selecting 8 experts per layer. It combines three layers of 512-token sliding-window attention with one layer of global attention. Its primary weights use a native `{-1, 0, +1}` ternary representation and are stored using different packing schemes in TQ1_0 or TQ2_0, rather than being produced by post-training quantization of a conventional BF16 model to an extremely low bit width.
The practical value of this merge is deployment compatibility. The official checkpoint, approximately 5.31 GB, can now use llama.cpp’s conversion pipeline, server, and surrounding toolchain without being tied to the vendor fork. The PR’s architecture test achieved `NMSE 8.75e-08`, and the author measured approximately 216 tokens/s for prefill and 88 tokens/s for generation on an Apple M4 CPU. These figures are not directly comparable with the model card’s claim of 218 tokens/s, as the latter used a different Apple Silicon runtime, and neither the test conditions nor the generation stages were fully aligned.
The review process also exposed numerical issues worth monitoring. TQ1_0 and TQ2_0 should preserve the same set of ternary weights when used with the same output head, yet different quantization operations and CPU/CUDA backends previously produced significant perplexity gaps. Changing the output head to Q4_K also caused a small quality loss. The merged implementation establishes the CPU architecture path as the baseline; it should not be taken as evidence that CUDA, Metal, or Vulkan already delivers equivalent speed and correctness. The PR also disclosed that some code was generated by an agent and subsequently reviewed and tested by humans. Engineering teams should wait for an official build, then validate speed, memory usage, and quality together using their own context lengths, backends, and task data.