本機推論
gemma4.c Reimplements Gemma 4 E2B Inference in Roughly 700 Lines of Pure C, Outperforming General-Purpose Frameworks in Single-Machine CPU Tests
The open-source gemma4.c project packs the tokenizer, Transformer, KV cache, sampling, and SIMD kernels into a single C file, with no external inference library required. In the author’s Ryzen 7 7700 tests, it outperformed llama.cpp, but the results depend on a fixed model, a single CPU backend, and a specialized data layout, so they should not be treated as a general performance conclusion.
The community project [gemma4.c](https://github.com/ryanssenn/gemma4.c) implements the complete text inference path for Gemma 4 E2B in roughly 700 lines of pure C, covering the tokenizer, model tensors, Transformer, KV cache, sampling, and CPU kernels. At runtime, it does not depend on PyTorch, GGML, or any other inference library; Python is used only for exporting weights and numerical validation. This allows developers to start at `main()` and trace, step by step, how a prompt passes through memory allocation, matrix operations, and attention before becoming the next token.
The project first uses `exporter.py` to convert a Hugging Face checkpoint into the fixed binary layout expected by the runtime. Matrix weights use int8, scale factors use FP16, and linear-layer inputs are also dynamically quantized to int8, while the remaining activations stay in float32. The model file is about 5GB, and at least 8GB of RAM is recommended. The kernels use OpenMP and AVX2, enabling AVX-512 VNNI when available. It therefore still requires a compatible x86 CPU and compiler toolchain and is not a portable, cross-architecture minimal implementation.
In twelve formal benchmark runs on an AMD Ryzen 7 7700, the author reports that gemma4.c achieved approximately 633 tokens per second for a 512-token prefill and 25 tokens per second for a 128-token decode. In the same test, llama.cpp Q8_0 reached approximately 262 and 23 tokens per second, respectively. For numerical validation, outputs at 2,388 positions from WikiText-103 were compared against a Transformers BF16 reference, producing a 96.5% top-1 logits agreement rate and a mean KL divergence of 0.005207. These results show that a highly specialized data layout and kernel fusion can eliminate some of the overhead associated with general-purpose graph schedulers.
However, [community discussion](https://www.reddit.com/r/LLMDevs/comments/1w0r1ro/i_implemented_a_modern_llm_runtime_in_700_lines/) also notes that the speed comes from “removing generality”: the project supports only one model architecture, one quantization format, and a CPU backend, with no dynamic graph, GPU support, batched serving, broad sampler selection, or multi-model compatibility layer. The next questions to watch are whether the reported numbers can be reproduced on other hardware, how KV cache memory behaves with long contexts, and whether the project can retain its readability and performance after adding support for a second architecture.