vLLM Community Proposes RDNA Tiling Optimization, Measuring About 17%–19% Higher Embedding Throughput on W7900

On September 22, the vLLM community proposed an attention kernel optimization for AMD RDNA GPUs: reducing the size of the key-value blocks processed at each step. In embedding-serving tests on the Radeon Pro W7900, throughput increased by approximately 17%–19%. What is currently public is a performance proposal; the speedup should not yet be considered available in an official release. [Community tests](https://github.com/vllm-project/vllm/issues/58060)

The technical focus is matrix tiling. The public code for vLLM 0.28.0 shows that the query and key-value dimensions share the same block size, while the number of warps is determined by the attention head dimension. The proposal reduces only the key-value block size to 32, leaving the other launch parameters unchanged. This confines the tuning to a single kernel, but hardware profiling evidence is still needed to establish whether smaller blocks reduce shared-memory pressure. [Kernel code](https://docs.vllm.ai/en/v0.28.0/api/vllm/v1/attention/ops/triton_prefill_attention/), [Proposal details](https://github.com/vllm-project/vllm/issues/58060)

Tests used a single GPU, BF16, and stock clock speeds. Median kernel speedups were approximately 2.6× on the W7900 and 2.3× on the R9700. Yet serving throughput on the latter increased by only about 3%–5%, showing that kernel speed does not translate directly into request-level performance. The author also explicitly limits the proposal to the attention path used for embeddings and reranking; text generation is outside its scope. [Test conditions and results](https://github.com/vllm-project/vllm/issues/58060)

Model configuration is equally important when reproducing the results. GTE-Qwen2-1.5B, used in the serving tests, is configured for non-causal attention, matching the proposal’s intended use. Having Qwen in the model name does not mean it follows a conventional autoregressive decoding path. Engineers should verify the configuration and backend actually loaded before comparing performance; merely changing the output to vectors does not establish that computation follows the same path. [Model configuration](https://huggingface.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct/blob/main/config.json)

The vLLM documentation groups tasks such as embedding and classification under pooling models, which provide vectors or scores through the corresponding interfaces. It also cautions that support for these models does not guarantee better performance than using Transformers directly. RAG systems evaluating this optimization should therefore also compare latency across the complete retrieval pipeline, rather than looking only at the attention kernel. [Official documentation](https://docs.vllm.ai/en/latest/models/pooling_models/)

Current end-to-end data comes from only one card per architecture. Half-precision FP16, multiple GPUs, and other RDNA models remain untested, and retrieval metrics also show small changes. [Known limitations](https://github.com/vllm-project/vllm/issues/58060) Next steps worth tracking include an upstream patch, independent retesting, and ranking consistency on Traditional Chinese corpora. Short queries, long documents, and high concurrency may each encounter different bottlenecks, so evaluations should retain tail-latency measurements and failed-request logs alongside average throughput. These checks are needed to determine whether the inference time saved justifies the cost of maintaining a custom kernel.