AI 晶片與資料中心
MTIA 300 Packages Twelve RDMA NICs and Collective Engines On-Chip, Bypassing the PCIe Communication Path
Meta’s MTIA 300 integrates 1.2 TB/s of network I/O and sixteen messaging engines directly into the accelerator, preventing collective operations from consuming the main compute array. For a 150B-parameter recommendation model, Meta reports a 3.9× reduction in communication time versus a reference GPU cluster, but has yet to provide complete system data sufficient to compare total training costs.
Meta has disclosed the system design of MTIA 300, its first in-house accelerator built for training large-scale ranking and recommendation models. Embedding tables can account for more than 99% of the parameters in these models, requiring hundreds of accelerators to frequently execute AllReduce, AllToAll, and AllGather operations. The bottleneck is therefore not merely matrix multiplication throughput, but whether data can move between chips and racks fast enough.
MTIA 300 incorporates two network chiplets in the package, each containing six 800 Gbps RDMA NICs, providing a combined 1.2 TB/s of I/O without traversing the traditional host-to-device-to-NIC PCIe path. The twelve Ethernet NICs can be reallocated between intra-rack scale-up and inter-rack scale-out connectivity, with maximum configurations of approximately 1 TB/s and 200 GB/s, respectively. Meta has also added an express doorbell mechanism that allows the work-request write itself to trigger a transfer, which it claims saves roughly 800 ns per operation.
A more consequential change is the inclusion of sixteen independent messaging engines. Each engine contains a RISC-V control core, a NIC interface, and a near-memory reduction unit. As a result, collective operations do not consume the primary compute resources executing GEMMs, as they typically do with GPU communication kernels. HCCL, co-designed with the hardware, compiles collectives into dependency-aware work subgraphs. Once instructions are placed in HBM, the device can schedule them autonomously, with graph integration available through `torch.compile`, PyTorch c10d, and torchcomms.
Meta reports that running GEMMs and collectives concurrently reduces compute throughput by less than 0.5%, compared with more than 20% for its reference GPU system. HCCL reaches 940 GB/s within a single rack, while communication time fell by 3.9× when training a 150B-parameter recommendation model across 40 accelerators. However, these results come from a specific internal model and topology, using a vendor-selected GPU baseline. Meta has not disclosed end-to-end training time, power consumption, chip cost, or yield. For now, MTIA 300 primarily demonstrates the potential of hardware-software co-design for recommendation systems; it cannot yet be considered a direct replacement for general-purpose LLM GPUs. Future areas to watch include whether the same architecture can handle inference, long-context workloads, and high-frequency agent traffic composed of small messages.