All Articles · SURL AI News

NVIDIA publishes multi-GPU video inference benchmarks, cutting generation latency to about 34 seconds on eight GPUs

NVIDIA ran Cosmos 3 Nano using Dynamo-Triton and TensorRT, reducing generation latency from 156.6 seconds on one GPU to 34.2 seconds on eight. The tests did not cover concurrent throughput or per-video costs, and quality validation used only sampled frames.

2026-09-22

硬體

Googlebook opens for preorders, integrating agent development tools and an isolated Linux environment

Google is bringing Antigravity, a Linux terminal, and on-device AI hardware to its new laptop platform. The pKVM protection modes across processor variants and NPU access from Linux still require product documentation and testing on actual devices.

2026-09-22

GitHub Repo

vLLM Community Proposes RDNA Tiling Optimization, Measuring About 17%–19% Higher Embedding Throughput on W7900

Community tests improved embedding-serving performance on AMD GPUs by reducing the key-value block size in an attention kernel. The change remains a proposal, and kernel speedups, overall throughput, and retrieval quality must be evaluated separately.

2026-09-22

GitHub Repo

Ultralytics 8.4.157 adds CUDA Graph replay, cutting YOLO26n FP16 test latency by about 20%

The release uses real-image calibration and CUDA Graph replay to reduce inference overhead. Gains vary with precision and batch size, while the SiLU rewrite mentioned in the release notes was withdrawn during final review.

2026-09-22

推論系統

SemIf demonstrates in-browser option-score readout, but decision probabilities still need separate calibration

This local inference experiment compares reading option logits directly with generating JSON token by token, letting developers measure the cost of output formats. Results remain sensitive to quantization, browser behavior, and test order, and option probabilities cannot be treated directly as measures of reliability.

2026-09-20

推論系統

AMD details MLPerf inference optimizations, boosting eight-GPU GPT-OSS serving throughput by about 38%

AMD improved GPT-OSS-120B benchmark throughput on MI355X GPUs by splitting attention kernels and adjusting scheduling. Public weights and reproduction instructions provide a starting point, but aspects of the environment configuration and model card still need clarification.

2026-09-19

GitHub Repo

vLLM Publishes Multi-GPU NVDEC Tests, Showing More Than Twice the Video Captioning Throughput of CPU Decoding

vLLM has published multi-GPU scaling tests of hardware video decoding, easing CPU bottlenecks in short-caption generation. Gains remain workload-dependent, and deployments must reserve GPU memory for decoding processes.

2026-09-19

AI 基礎設施

AIPerf Reworks LLM Benchmarking with Multiple Processes, but Streaming Metrics Still Require Token-Counting Checks

NVIDIA details AIPerf, the successor to GenAI-Perf, which brings load generation, results processing, and traffic replay into a multiprocess architecture. It offers more comprehensive measurements of inference services, but migration compatibility and streaming token counts still affect how results should be interpreted.

2026-09-19

AI 研究

SoL-Pi cuts agent costs with four mechanisms, retaining about 94% of baseline EdgeBench scores

NVIDIA and collaborators used automated experiments to optimize an agent harness, reducing token traffic by roughly 45% to 49% across two models. The public extension preserves raw evidence for retrieval, but the full configuration still involves trade-offs in scores and task completion rates.

2026-09-19

AI 安全

ALIBI misleads LLM malware classifiers with fake product descriptions; verification prompts still leave errors

The study adds fabricated context that static analysis can extract without changing program execution behavior. The results show that models may lower their risk assessments by trusting unreliable descriptions, even without following malicious instructions.

2026-09-19

Needle 3 supports multiple depths with a single set of weights, shrinking on-device tool models to 8–29 MB

Cactus derives models with two to twenty layers from the same weights for on-device tool calling and structured extraction. Official comparisons are limited to narrow tasks after fine-tuning, and Chinese is not yet listed as a supported language.

2026-09-19

推論系統

Anthropic Releases 36 Biological Model Acceleration Packages, Incorporating Numerical Consistency Into Execution Modes

Developed with Claude’s assistance, the 36 packages use shared GPU kernels to accelerate biological models and reduce memory requirements. Anthropic distinguishes between identical-output and approximate acceleration, but does not commit to ongoing maintenance of the public release.

2026-09-19