推論系統
llama.cpp b10472 Fixes AMD APU Memory Detection, Preventing Strix Halo from Being Assigned Nonexistent Available Capacity
The new release stops overriding GPU memory information with the system’s `MemAvailable` value in Linux HIP builds and instead trusts `hipMemGetInfo`. The fix targets AMD APUs with configurable UMA carveouts and reduces the risk of the model loader overestimating capacity and failing later.
llama.cpp released b10472 on August 17, fixing unified memory detection for AMD APUs on Linux. UMA logic previously added to support NVIDIA DGX Spark treated the operating system’s `MemAvailable` value as the capacity available to the accelerator. This approximation is unreliable on AMD APUs such as Strix Halo because the BIOS can divide unified memory into different system RAM and VRAM carveouts. The amount of main memory available to Linux is not the same as the graphics memory that the HIP runtime can actually allocate.
For HIP builds, b10472 skips this override, allowing AMD APUs to continue using the free and total values reported by `hipMemGetInfo`. This is not a benchmark optimization but a resource-scheduling correctness fix: llama.cpp uses available capacity to determine tensor offloading, whether a model will fit on the GPU, and KV cache and batch configurations. If the value is overestimated, a service may initially accept an oversized model or context and then encounter an OOM error during allocation. Systems with small carveouts, containerized deployments, and high shared-memory pressure are especially susceptible.
The merge discussion noted that BIOS settings can leave Strix Halo with only about 32 GB of system RAM while assigning the rest to the GPU; `/proc/meminfo` alone cannot reconstruct this boundary. After upgrading, engineering teams should re-record the HIP free and total values at startup, the number of layers that can actually be offloaded, and the maximum KV cache size. The fix should not be interpreted as a sudden increase in available memory. This change explicitly covers only the Linux HIP path; maintainers confirmed that ROCm memory reporting works differently on Windows. The release also provides no comparative throughput or OOM-rate data, so further validation across different BIOS carveouts, kernels, and ROCm versions is still required.