推論系統
Daedalus-150M Replaces Two-Thirds of Attention Layers With Convolutions, Accelerating CPU Decoding by 1.76× at 2K Context
Daedalus-150M is designed specifically for single-user, 4-bit CPU inference, allowing most network layers to avoid repeatedly reading an ever-growing KV cache. Preliminary results indicate that its speed advantage increases with context length, but the model is limited to 2K context, and its performance figures still await independent reproduction across different hardware.
Daedalus-150M released its paper, code, and weights on August 20. Rather than compressing a standard Transformer for CPU deployment after the fact, its architecture was derived from memory-access costs. The full model contains 18 blocks, only six of which use full attention. The remaining 12 layers use depthwise convolutions with a kernel size of 3, with each layer retaining a fixed state covering only two time steps. The attention layers use GQA with 12 query heads and four KV heads.
This configuration directly reduces the amount of cache that must be reread during autoregressive decoding. The authors calculate that, at a context length of 2,048 tokens, Daedalus reads approximately 12.6 MB of cache for each generated token, compared with 25.2 MB for a 24-layer, full-attention reference architecture. In tests using 4-bit weights and eight CPU threads, Daedalus generated 1,112 tokens/s with an empty context, an advantage of only 1.20×. At 2,048 tokens, it still achieved 739 tokens/s, while a parameter-matched model trained with the same data and recipe fell to 420 tokens/s, widening the gap to 1.76×. Its largest reported advantage over external models of a similar size was 2.08×.
The model was trained from scratch on 59.9 billion tokens. It averaged 47.31 across five commonsense benchmarks, outperforming several earlier models with roughly 130 million to 160 million parameters, but remaining behind SmolLM2-135M, which was trained on two trillion tokens and averaged 51.2. The project provides Apache 2.0-licensed code, GGUF files, and Hugging Face weights, along with evaluation logs and parameter-matched ablations to facilitate reproduction.
The limitations are equally clear: the model currently supports only English and a 2,048-token context window, and the reported results come from a single training seed. Q4_0 quantization causes an approximately 6% perplexity degradation, around 48% of convolution channels appear inactive, and the oversized 49,152-token vocabulary accounts for roughly 23% of the parameters. The next steps should include testing across different x86 and ARM CPUs, extending the context length, and determining whether pruning inactive channels can preserve the same scaling curve for decoding speed.