Back Home

GitHub Repo

Ollama Preview Fixes Vulkan iGPU Loading Strategy to Reduce Shared Memory Allocation Failures

On November 10, Ollama merged a fix that switches Vulkan integrated GPUs to mmap model loading, including it in v0.40.17-rc0. The change targets load failures caused by duplicate use of shared memory; compatibility with actual hardware remains to be verified.

Mattruffoni · CC BY-SA 4.0 · Image source
zh-Hant

Ollama merged a model-loading fix for Vulkan integrated GPUs on November 10 and included it in the v0.40.17-rc0 preview released that evening. The update addresses a memory allocation problem during model startup, with direct implications for developers running local models on AMD and Intel integrated GPUs. Preview release notes

Maintainers said that without mmap, tensors kept on the CPU are placed in a Vulkan host buffer, which shares the device memory budget with the model weights. On integrated GPUs that share system memory, this can nearly double usage, potentially causing locked memory allocation failures or exception codes in Windows AMD environments. Direct I/O also cannot bypass this buffer, so Vulkan needs a separate loading strategy. Maintainer’s explanation

The code now passes --load-mode mmap to the underlying llama-server when it encounters a Vulkan device marked as integrated. Integrated CUDA and ROCm devices on Linux continue to use direct I/O. New tests also explicitly check that this rule does not apply to discrete Vulkan graphics cards, while the integrated GPU path prioritizes mmap even when the user option disables it. Patch and tests

A related earlier report says that, on the same hardware and with the same model, a Radeon 780M might fail to start a larger model after an upgrade, while smaller models still worked. This shows that loading failures cannot be judged solely by model file size or available system RAM; the backend’s allocation method also affects usable capacity. The fix is now linked to the closure of that report. Issue record

Official documentation confirms that Vulkan supports Windows and Linux and provides settings for selecting a GPU or disabling the backend. When testing the preview, engineering teams should hold the model, context length, and driver version constant, then compare whether loading succeeds and the memory peak. Public evidence is not yet sufficient to conclude that all integrated GPUs have recovered, and there are no broadly applicable data showing improved generation speed. Hardware support documentation

Sources

  1. Ollama v0.40.17-rc0 發布紀錄
  2. llm: use mmap for Vulkan integrated GPUs
  3. Vulkan 載入策略修補與測試差異
  4. AMD Radeon 780M Vulkan regression in Ollama >=0.32.10
  5. Hardware support