硬體

AI 晶片、GPU、資料中心、邊緣裝置與硬體安全。

FreeToken Dynamically Splits Expert Computation Across CPU and GPU, Enabling 284B MoE Inference on a Single RTX 5090

Instead of offloading MoE experts according to fixed rules, FreeToken dynamically divides work between CPU execution and GPU caching based on measured memory and PCIe bandwidth. The authors report that an RTX 5090 can run a 284B model at 22–25 tokens per second, though testing was conducted by the development team and the full weights still require substantial system memory.

2026-08-29

硬體

MTIA 300 Packages Twelve RDMA NICs and Collective Engines On-Chip, Bypassing the PCIe Communication Path

Meta’s MTIA 300 integrates 1.2 TB/s of network I/O and sixteen messaging engines directly into the accelerator, preventing collective operations from consuming the main compute array. For a 150B-parameter recommendation model, Meta reports a 3.9× reduction in communication time versus a reference GPU cluster, but has yet to provide complete system data sufficient to compare total training costs.

2026-08-29

AI 安全

Google Tests Double-Blind Model Evaluation in an H100 Secure Enclave, Keeping Model Weights and Confidential Test Sets Hidden from Each Other

Google DeepMind, MLCommons, and other organizations placed Gemini 2.5 Flash Lite and an undisclosed safety test set inside the same attestable secure enclave, with each party receiving only restricted results. The design reduces the risk of benchmark contamination, but trust still rests on hardware root keys, the trusted computing base, and cross-organizational code review.

2026-08-29

硬體

AWS Receives First Vera CPU/Vera Rubin Systems, Plans to Add 2 Million NVIDIA GPUs Over Two Years

AWS has received its first Vera CPU servers and Vera Rubin GPUs, while the two companies plan to deploy an additional 2 million NVIDIA GPUs from 2027 to 2028. The collaboration also incorporates NVLink Fusion, NVHBM, Nitro, and EFA into heterogeneous rack designs, though most capacity and performance figures remain future commitments.

2026-08-29

AI 軟體／文件處理

Cohere Parse 5 Uses a 2.3B Vision Model to Process Enterprise Documents, but Its Benchmark Score Excludes Charts and Layout Localization

Parse 5 converts PDFs, presentations, and images into Markdown while preserving tables and reading order, with API pricing of $1.50 per 1,000 pages. Its officially reported ParseBench score of 79.2 covers only three of the benchmark’s five evaluation dimensions and should not be treated as a complete document-understanding score.

2026-08-29

推論系統

TensorRT Model Connect Compiles Model Checkpoints into Native C++ Bundles, but the Interface Is Not Yet Finalized

NVIDIA has released TensorRT Model Connect, which uses Python build tools to package supported Hugging Face checkpoints into TensorRT bundles loadable from C++. The project claims support for 84 model families and 221 end-to-end manifests, but it remains in nightly development, and inclusion in the support matrix does not mean every hardware configuration has been validated.

2026-08-29

GLM-5.3 Releases 744B MoE Weights, but Million-Token Deployment Still Requires Data-Center-Class Resources

Two weeks after launching the API, Z.ai has released the GLM-5.3 weights. The model retains GLM-5.2’s base architecture, with capability gains driven primarily by post-training for agentic coding and cybersecurity. The FP8 files still total about 756GB, and the custom license and vendor-led evaluations mean that performance, safety, and legal validation remain necessary before adoption.

2026-08-29

AI 基礎設施

ROCm 10 Brings Agentic Kernel Optimization to General Availability, but the Upgrade Also Introduces ABI Breakage and Known Issues

AMD has released ROCm 10 and made ROCm.AI—integrating Hyperloom, AMD Skills, and a unified CLI—generally available. The company claims average inference and training performance gains of 3.3× and 2.4× on the same hardware, but those figures cover the entire optimization workflow and should not be interpreted as gains from simply upgrading the runtime.

2026-08-28

推論系統

Atlas Takes On vLLM With a Pure Rust/CUDA Inference Engine on a Single GB10, Delivering Up to 22.5% Higher Throughput Under High Concurrency

Atlas has published reproducible Qwen3.8-27B-NVFP4 benchmarks showing it outperforming vLLM at concurrency levels from 1 to 128 on DGX Spark. The results cover only one model and a short-input, long-output workload, so they cannot yet be generalized to typical data center deployments.

2026-08-28

推論系統

vLLM Supports 16K Embeddings on Ironwood TPUs, StepPool Preserves Chunked Prefill State

Google and vLLM have added tensor alignment, compilation warmup, and pooling state that accumulates across chunks for Qwen3 embedding models. Official results report roughly 84,000 tokens/s on four Ironwood TPUs, but no cost-equivalent GPU comparison or real-world retrieval quality testing has been provided.

2026-08-28

硬體

NVIDIA Moves the Memory Controller Into the NVHBM Base Die, Freeing Up to 25% of Custom AI XPU Area

Using a custom base die and PHY, NVHBM moves some memory-interface logic off the compute die and integrates it into NVLink Fusion. NVIDIA claims up to 30% more bandwidth and 15% lower memory power consumption than standard HBM4E, but has not yet published measurements from production silicon.

2026-08-27

硬體

OpenAI Jalapeño Delivers 1.5–1.9× Higher Throughput per Watt on Three Open Models, but Long-Context Agent Workloads Remain Untested

OpenAI has published the first benchmark results for its in-house inference ASIC, which uses a single accelerator architecture for both prefill and decode. SemiAnalysis observed and verified the testing process on site, but the data was still supplied by OpenAI, while production silicon and AgentX results have yet to be validated.

2026-08-25