NVIDIA adds NVHBM high-bandwidth memory to NVLink Fusion

Summary

NVIDIA has expanded its NVLink Fusion connectivity platform with NVHBM. This high-bandwidth memory variant moves the memory controller from the XPU die to the HBM base die. The company claims up to a 30% increase in bandwidth compared to HBM4E and up to a 25% increase in compute chip area. Amazon's Annapurna Labs unit is highlighted as the first to integrate NVHBM into its Trainium4 chips within NVIDIA's rack-scale architecture.

Why it matters

Memory bandwidth and power consumption are dominant bottlenecks in trillion-parameter and agentic AI workloads. Therefore, moving the controller off the compute die provides a structurally significant efficiency gain. Strategically, this move deepens NVIDIA's dominance in the rack-scale connectivity layer. Even hyperscalers developing their own XPUs are forced to adopt NVIDIA's memory interface and switching infrastructure to participate in this space. This reinforces a 'pick and choose' ecosystem that is nominally open but practically tightly coupled.

Highlights

• Moving the memory controller to the 3D HBM stack frees up to 25% of the XPU die area for compute.
• Amazon Annapurna Labs, as NVHBM's first partner, is pairing the technology with Trainium4 in a shared rack-scale design.
• NVIDIA positions NVHBM as a multi-vendor 'standard' to reduce the burden of qualification processes for custom XPU designers.

Critical take

Every figure in the headline is qualified with 'up to.' The 30% bandwidth and 15% power consumption data are presented without a controlled workload or comparison baseline, making independent verification difficult. The 'horizontally open' framework also deserves scrutiny. By making NVHBM a prerequisite for NVLink Fusion's rack-scale advantages, NVIDIA is effectively turning memory interface preference into an ecosystem lock-in, even among its largest customers, including direct competitor AWS.