推論系統
AMD Publishes Kimi-K3 Tests Across Three Engines; Reproduction Requires Checking Workloads and Image Versions
A new article details inference testing on eight MI350X GPUs using data collected in July. The README at the specified commit differs from the article on workloads, while differences in engine configurations also limit performance comparisons.
On September 22, AMD published its testing procedures for Kimi-K3 on vLLM, SGLang, and ATOM, detailing inference configurations, container versions, and measurement methods for a single node with eight MI350X GPUs. The data was collected on July 29 and represents a snapshot of the software stack at that time; the news here is the public disclosure of the methodology and its limitations. [AMD technical article](https://rocm.blogs.amd.com/artificial-intelligence/kimi-k3-mad/README.html)
The workflow uses MAD’s model list to link container builds, launch scripts, and configuration files, then runs them through madengine to produce results in a common format. madengine saves build manifests and allows builds and execution to be performed separately, making it easier to reuse the same image across nodes. For engineering teams, this structure helps trace performance regressions to configuration and version differences. [madengine repository](https://github.com/ROCm/madengine)
The Kimi-K3 integration adds a model entry, dedicated containers, and scripts for each engine, using eight-way tensor parallelism. The repository documentation states that the model weights occupy approximately 1.56 TB, so sufficient model cache space must be reserved before rerunning the tests. The configurations target the architecture used by the MI350X and MI355X, but the article’s actual tests used only MI350X hardware. [Integration record](https://github.com/ROCm/MAD/pull/186), [documentation at the specified commit](https://github.com/ROCm/MAD/blob/a20c885/benchmark/kimi_k3/README.md)
AMD says the workload used 8,192 input tokens and 1,024 output tokens, with measurements taken at different numbers of concurrent requests and prefix caching disabled. However, the README at the specified commit still lists the vLLM example with 1,024 tokens for both input and output, while the concurrency ranges listed for the other engines also differ. At a minimum, this shows that the documentation is not yet aligned. Before running the tests, users should check the fully expanded commands rather than assume that copying the entry-point command will produce the same workload. [Article methodology](https://rocm.blogs.amd.com/artificial-intelligence/kimi-k3-mad/README.html), [README examples](https://github.com/ROCm/MAD/blob/a20c885/benchmark/kimi_k3/README.md)
The article also records that container tags pointed to different hashes during the build process, so reproducing the tests requires pinning image digests. Each measurement point was run only once, with no warm-up or estimate of variability. Memory budgets and KV cache precision were not standardized across engines, and the tests did not evaluate answer quality. [Measurement limitations](https://rocm.blogs.amd.com/artificial-intelligence/kimi-k3-mad/README.html)
From an engineering perspective, a common report format can reduce the effort needed to compile results, but it does not automatically standardize timing boundaries, prompt-length sampling, or stopping conditions. If the workflow is integrated into continuous integration, raw output and server parameters should be retained to distinguish software regressions from shifts in test conditions. Before selecting an engine for production serving, teams still need repeated measurements and validation using real-world prompt lengths, latency requirements, and quality checks. Total token throughput must also be interpreted separately from generation speed measured using output tokens alone.