本機推論
Local LLM Testing on M4 Max: Faster Decoding Than GB10, but Full Responses Remain Constrained by Prefill
A new round of similarly priced local AI testing shows that the M4 Max, with 546GB/s of unified memory bandwidth, leads the GB10 and Strix Halo in per-token decoding across three quantized models. The GB10 still has the advantage in prompt prefill and overall energy efficiency, demonstrating that tokens/s alone can misrepresent actual response times.
On July 30, Tom’s Hardware used `llama.cpp` to compare the Apple M4 Max, Nvidia GB10, and AMD Ryzen AI Max+ 395, testing four-bit quantized versions of Qwen 3.6-35B-A3B, Gemma 4 12B, and gpt-oss-120b. The tested Mac Studio featured a 40-core GPU, 128GB of unified memory, and 546GB/s of memory bandwidth. Its price at the time of testing was about $3,699, close to the DGX Spark’s $3,999 suggested retail price, making the comparison more relevant to desktop deployment decisions than a simple chip-specification comparison.
The results show that two bottlenecks in local LLM inference should not be conflated. Prefill processes the entire prompt in parallel, making compute throughput, core count, and software-kernel optimization more important. Decode, by contrast, generates tokens sequentially through every model layer and must repeatedly stream weights from memory, so it is generally more constrained by memory bandwidth. The M4 Max led the other two systems in decoding throughput across all three models and at different context lengths, with an especially pronounced advantage on the Qwen MoE model. However, the GB10 was significantly faster at prompt processing. In a complete gpt-oss-120b workload with a 2,048-token input and a 1,024-token output, the GB10’s faster prefill resulted in lower overall completion time and energy consumption than the M4 Max.
These findings are particularly important for RAG, coding agents, and long-context services. Chat workloads with short prompts and long outputs may favor high-bandwidth platforms. For agents that repeatedly ingest large codebases, documents, or compressed multi-turn histories, prefill speed and KV cache growth may instead dominate latency. Engineering teams should measure time to first token, decode throughput, total turn completion time, power consumption, and maximum supported model size rather than basing purchasing decisions on a single tok/s ranking. One limitation is that this remains an independent media benchmark: the quantized model files, `llama.cpp` commit, degree of Metal/CUDA optimization, and available-memory configuration can all affect the results. The memory configurations Apple currently offers for the M4 Max may also differ from the tested 128GB system.