AI 晶片與資料中心基礎設施
NVIDIA Moves the Memory Controller Into the NVHBM Base Die, Freeing Up to 25% of Custom AI XPU Area
Using a custom base die and PHY, NVHBM moves some memory-interface logic off the compute die and integrates it into NVLink Fusion. NVIDIA claims up to 30% more bandwidth and 15% lower memory power consumption than standard HBM4E, but has not yet published measurements from production silicon.
On August 26, NVIDIA added NVHBM to NVLink Fusion, offering cloud providers designing custom AI accelerators an NVIDIA-defined HBM base die, physical interface, and memory controller. Conventional designs place the controller on the XPU compute die; NVHBM instead integrates it into the logic die at the bottom of the HBM stack, shortening the data path while reclaiming advanced-process die area previously occupied by memory I/O for matrix computation, cache, or other specialized units.
According to NVIDIA’s design targets, NVHBM can deliver up to 30% higher memory bandwidth than standard HBM4E, reduce HBM power consumption by 15%, and free up as much as 25% of the XPU die area. The company further estimates that reallocating this area to compute units could improve end-to-end XPU performance by approximately 30%. These figures describe specific integration configurations and simulation or internal validation results; they cannot be applied directly to every model, package, or workload with a different degree of memory bottlenecking.
NVHBM also has implications for supply-chain and platform integration. NVIDIA says multiple memory vendors will offer validated implementations based on a common design, reducing the work customers must do to design and validate base dies independently. Amazon subsidiary Annapurna Labs is the first publicly disclosed partner and plans to use the technology with the NVLink scale-up architecture in future custom chips. However, neither company has disclosed the Trainium generation involved, memory capacity, number of stacks, production timeline, or actual system configuration.
For accelerator teams, this is not a plug-and-play, general-purpose HBM upgrade. It is a semi-custom component that must be jointly integrated during the chip, package, firmware, and NVLink rack design stages. Key areas to watch include the list of memory suppliers, yield and cooling costs, and third-party benchmarks for inference KV cache workloads, expert parallelism, and large-scale training communication. Until physical products ship, the claimed bandwidth and performance gains should still be treated as vendor targets.