Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
AI 基礎設施
NVIDIA has disclosed NVLink 6’s multilayer fault-tolerance design, extending from physical-layer bit-error correction to the NMX Controller and Dynamo inference processes. Internal B200 testing reduced the time required to restore service on two workers after a failure from 283 seconds to 7.3 seconds, though checkpointing tools and NCCL edge cases are still evolving.
最新模型
Mozilla has added Mistral Small 4 to the Smart Window beta in the United States and Canada, expanded the French-language version to France, and retained other model options. The 119B-parameter MoE model offers a long context window and reasoning modes, but zero data retention does not mean local inference, and the boundaries governing the transmission of tab content still require real-world testing.
代理工具
Copilot’s shared agent runtime has migrated from TypeScript/Node.js to native Rust and can be embedded directly into SDKs for six languages through a C ABI. GitHub’s stress tests showed session throughput rising to 120 per second, though the figures come from internal tests that excluded model and network latency.
最新模型
Rather than estimating only conditional labels, LimiX-2 learns the joint structure of features and targets from synthetic causal data. The team has released the weights and inference code, but the model is restricted to non-commercial use, and its leading results come primarily from tabular benchmarks run by the authors.
代理工具
The new VS Code toolkit can deploy hosted agents from private container registries configured with a connection, while also adding an AMD NPU build of Fara 7B and a quantized Aion adapter. Windows ML has been upgraded to `windowsml` 2.3, requiring existing projects to review their execution providers and dependencies.
AI 基礎設施
Unity’s team introduced FlashVector, an agent that searches for performance bottlenecks across GPU kernels, computation graphs, Triton servers, and feature services. Production tests showed throughput gains of up to 2×, but the code, modification traces, and complete cost data have not been released.
推論系統
By combining 4-bit KV, component residency management, and cross-request state transitions in MLX, JustFit enables a 27B model to perform long-context inference within 24 GiB of unified memory. The reported 6.93× capacity increase comes from a single-machine, model-specific test, and some historical environment details and checkpoint identities cannot yet be fully reproduced.
推論系統
MLCommons is no longer testing only individual models. Inference 6.1 incorporates retrieval, reranking, generation, and multi-turn agent workflows. The new release also introduces an API-based client/server harness, signaling that data center benchmarking will shift toward the more production-like MLPerf Endpoints suite.
AI 基礎設施
SiFive booted ROCm 10 on a BigSky server powered by a P870-D RISC-V CPU, with a Radeon AI PRO R9700 running Gemma4-E2B inference. The demo shows that non-x86, non-Arm hosts can connect to AMD’s GPU software stack, although no performance data, reproducible image, or formal support scope has yet been released.
推論系統
vLLM and Red Hat AI have released a 5-billion-parameter DSpark draft model for Kimi K3, along with a complete recipe for extracting and transferring hidden states across nodes. In official tests, single-user generation speed increased from about 110 to 435 tokens/s, though the results depend on a GB300 cluster and a specific inference workload.
最新模型
Using Nemotron 3 Super as its foundation, Salesforce trained Koa, its first CRM reasoning model, through synthetic enterprise scenarios, SFT, and GRPO. It delivers clear improvements in multi-turn tool use, but the paper acknowledges that its overall performance still trails the strongest frontier models, while the official claim of “three times fewer errors” comes from Salesforce’s own CRM benchmark.
搜尋與推薦系統
Google Research distills a set-level search strategy learned through reinforcement learning into a lightweight diffusion model that can generate multiple vectors in parallel. Experiments claim a 12× to 20× speedup over autoregressive expansion, but the evidence so far is concentrated on fashion data and a non-public music dataset.