GitHub Repo
Ollama Community Reports Vulkan Cache Performance Anomaly, with Repeated Requests Slowing on Specific AMD Configurations
An October 22 report from the Ollama community says that, on a specific AMD RADV setup, enabling prompt caching caused generation speed to drop sharply on the second identical request. A contributor could not reproduce the issue on another AMD GPU; the root cause and scope remain unclear.
A performance report on Ollama’s Vulkan inference path is worth tracking for local deployment teams. An issue filed on October 22 reports that, with Ollama 0.40.8 on Ubuntu using a Radeon RX 6600 XT and Mesa RADV, generation speed on the second identical prompt fell from 78.50 tokens per second to 2.64. The model remained fully offloaded to the GPU, and the process did not crash. These results come from a specific environment and cannot yet be generalized to all AMD devices. Source: issue report
The reporter used qwen3:4b, a 4096-token context, and a maximum of 64 output tokens, calling the native generation API directly. After setting the service environment variable LLAMA_ARG_CACHE_RAM to 0 and restarting, three consecutive requests each maintained a speed of about 78 tokens per second. However, this setting disables both the host-memory prompt cache and the idle-slot cache. The test did not isolate their effects, so it cannot establish that either cache mechanism alone is the root cause. Source: reproduction steps and limitations
For engineering teams, this is a reminder to include repeated requests after the model is loaded in performance acceptance testing. Ollama’s official API reports model load, prompt processing, and generation durations separately. According to the documentation, generation throughput can be calculated as eval_count ÷ eval_duration × 10⁹, then assessed alongside total duration. This helps distinguish load latency, prompt processing, and decoding speed, instead of relying only on end-to-end wait time to identify a bottleneck. Source: generate API documentation
There is also contrary evidence: a project contributor tested a different AMD RADV GPU with the same Ollama version and saw both requests run at about 78 tokens per second, with no slowdown. The official documentation also notes that Linux Vulkan can use different driver implementations and that a GPU can be specified. Follow-up should track independent reproductions on the same hardware, tests that isolate the cache mechanisms, and an official fix. For now, disabling the cache is best treated only as a comparison experiment in affected environments. Source: contributor test, hardware support documentation