AI 晶片與資料中心
MTIA 300 Integrates 12 RDMA NICs and Collective Communication Engines On-Chip, Avoiding PCIe and Compute-Core Contention
Meta has disclosed the system design of MTIA 300, which accelerates recommendation-model training with built-in NIC chiplets, near-memory reduction units, and compiled HCCL. The company says communication time across 40 accelerators was reduced to roughly one-quarter that of a reference GPU cluster, but provided no comparisons of cost, power consumption, or general-purpose LLM training.
Meta has disclosed further architectural details of its in-house MTIA 300 accelerator. Rather than pursuing the highest floating-point throughput of a general-purpose GPU, the design targets communication bottlenecks in large recommendation and ranking models. More than 99% of these models’ parameters may reside in embedding tables, and training frequently performs AllReduce, AllToAll, and AllGather operations across hundreds of accelerators. If collective communication is still executed by GPU kernels, it competes with matrix operations for the same compute cores.
MTIA 300 incorporates two networking chiplets in the package, containing a total of 12 custom 800Gbps RDMA NICs and delivering aggregate I/O bandwidth of 1.2TB/s without routing data through PCIe. The same NICs can be reallocated between 1TB/s intra-rack scale-up connectivity and 200GB/s inter-rack scale-out connectivity. The chip also includes 16 independent message engines. Each engine integrates a RISC-V control core, a NIC interface, and a near-memory reduction unit capable of processing 128 bytes per cycle, for combined reduction throughput exceeding 2.8TB/s.
The accompanying HCCL implementation does not rely on the host to drive communication step by step at runtime. Instead, it compiles each collective into a work-queue subgraph with explicit dependencies, which the message engines then execute autonomously. It can integrate with computation graphs through PyTorch `c10d`, TorchComms, and `torch.compile`, and it also provides paths for one-sided communication and device-triggered collectives. Meta reports peak intra-rack bandwidth of 940GB/s. When collective communication and large GEMM operations run concurrently, compute throughput declines by less than 0.5%. When 40 MTIA 300 accelerators ran a production recommendation model with 150B parameters, total communication time was 1/3.9 that of the reference GPU cluster.
These figures remain vendor-reported results obtained using Meta’s own model, topology, and software stack. MTIA 300 includes 216GB of HBM3E, but Meta has not disclosed the chip’s power consumption, total system cost, process node, or which GPU served as the comparison baseline. Nor has it demonstrated that the accelerator is suitable for matrix-compute-heavy frontier LLM training. Engineering teams should next examine HCCL availability, cross-framework support, and whether the integrated networking design can maintain the same advantages under the small-message, high-concurrency workloads typical of inference.