Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
AI 基礎設施
NVIDIA details AIPerf, the successor to GenAI-Perf, which brings load generation, results processing, and traffic replay into a multiprocess architecture. It offers more comprehensive measurements of inference services, but migration compatibility and streaming token counts still affect how results should be interpreted.
AI 研究
NVIDIA and collaborators used automated experiments to optimize an agent harness, reducing token traffic by roughly 45% to 49% across two models. The public extension preserves raw evidence for retrieval, but the full configuration still involves trade-offs in scores and task completion rates.
GitHub Repo
The release brings node health, model replicas, and batch lifecycle operations into the cluster dashboard while standardizing authentication for private sources. A new benchmark tool enables regression measurements but does not cover time to first token or capacity under high concurrency.
最新模型
A new technical report reveals how cross-layer cache reuse, FP4 storage, and local state reconstruction reduce deployment costs for long-context agents. The compression figures do not represent total GPU memory requirements, and the quality limits of approximate reconstruction still need validation.
推論系統
Developed with Claude’s assistance, the 36 packages use shared GPU kernels to accelerate biological models and reduce memory requirements. Anthropic distinguishes between identical-output and approximate acceleration, but does not commit to ongoing maintenance of the public release.
阿里巴巴推出接受文字、影像、音訊與影片的 Qwen3.8-Omni-Flash，並支援多聲道空間音訊。開發者可透過既有 API 串接工具與快取，但語音生成、素材處理成本及完整代理成功率仍須分別評估。
推論系統
Z.ai has detailed an engineering approach involving GLM-5.3 in inference system development, feeding numerical tests, execution traces, and local benchmarks into agent iteration. Upstream code confirms a long-context precision fix, but the new path is disabled by default and cannot independently substantiate the claim of threefold overall throughput.
最新模型
PrismML uses ternary weights and orthogonal rotations to compress Qwen3.8-27B, releasing the model and inference tools. The company claims to retain roughly 98.2% of overall benchmark performance, but differences in evaluation methodology and runtime compatibility still require careful checking.
AI 安全
The llama.cpp RPC server could replay cached computation graphs that still pointed to freed buffers, allowing remote clients to gain read/write capabilities and execute code. b11000 discards the associated graph state when a buffer is freed, but RPC still should not be exposed directly to untrusted networks.
推論系統
Novita AI and vLLM have open-sourced Chord, a W4A16 CUDA operator tuned for Kimi K2.x serving workloads on H200, B200, and B300 GPUs. Public benchmarks show performance of up to 2.15× that of Humming, but the figures cover only the MoE layer kernel and cannot be directly translated into end-to-end throughput.
硬體
OpenVINO 2026.4 adds Multi-Token Prediction speculative decoding for Gemma 4, Qwen3.5, and Qwen3.6, while enabling VTune to analyze inference across CPUs, GPUs, and NPUs. The release expands local model coverage, but several NPU dynamic-shape and model-unloading features remain in preview, and Intel has not provided standardized public acceleration figures.
代理工具
A team from Google Research and CMU separates strategy exploration, proof decomposition, local repair, and global verification into a multi-agent workflow. Cross-model selection reaches 71% on TCS-Bench, but the main harness has not been fully released, while automated grading and large inference budgets limit comparability.