硬體

AI 晶片、GPU、資料中心、邊緣裝置與硬體安全。

TensorRT Model Connect Enters Public Preview: Package Hugging Face Models for Native C++ Inference Without ONNX

NVIDIA has open-sourced TensorRT Model Connect, which uses two commands to turn supported Hugging Face or local checkpoints into versioned bundles that can be handed off to C++ applications. It shortens the model-porting path, but remains a reference implementation; support and performance must be validated for each model, precision, and hardware configuration.

2026-08-19

硬體

PTXBench Tests LLMs Writing H100/B200-Specific PTX Directly; No Model Reliably Matches State-of-the-Art Kernel Libraries

PTXBench checks whether generated kernels are correct, whether specified instructions actually execute, and how their performance compares with cuBLAS, cuDNN, and FlashInfer. The results show that generating instructions for new architectures does not necessarily translate into effective acceleration, with B200 attention kernels proving especially difficult.

2026-08-19

代理工具

StagedWorkspace Uses Content Hashes to Synchronize Agents’ Parsed Views and Native Files, Boosting OfficeQA Pass@1 by 8.3 to 12.1 Points

The new system binds a document agent’s search results, native files, diff views, and final submissions to explicit versions, preventing the agent from reading stale parsed content while editing a newer file. Ablations within a fixed framework show that providing both parsed and native views outperforms either view alone, though the full codebase and cross-framework reproductions are still needed.

2026-08-19

推論系統

MoNe Writes Long Context into Test-Time Neural Memory, Reducing Total FLOPs for 128K Inference by 81%

Qualcomm AI Research adds per-layer fast weights to a frozen Transformer, absorbing context in 512-token segments so queries no longer need to read the source text. In 128K experiments, Qwen2.5-0.5B maintained fixed memory usage and outperformed ICL and RAG, though the evidence is currently limited to three synthetic retrieval tasks.

2026-08-19

硬體

rl-triton Unifies Seven Reinforcement Learning Return Computations as GPU Scans, Delivering Up to a 5.70× Speedup for Full Calls

The open-source rl-triton library rewrites credit-assignment recurrences such as GAE, V-Trace, and Retrace as Triton associative scans, reducing sequential dependency depth from O(T) to O(log T). In tests with 4,096 environments and 128 steps, it was 1.6× to 5.70× faster than an already vectorized torch.compile baseline, although end-to-end PPO speedup for large policies was only about 1.02×.

2026-08-19

推論系統

FluxBin Uses Binary Bases with a LUT CUDA Kernel, Reports Up to 5.92× Faster Inference on a Single A100

Rather than expanding ultra-low-bit weights back into FP16, FluxBin uses binary patterns to directly index precomputed partial sums. The paper reports up to a 5.92× speedup and a 10.19× improvement in energy efficiency on a single A100, but its public code link remains inaccessible.

2026-08-18

推論系統

FreeToken Dynamically Splits MoE Cache Misses, Runs a 35B Model at 39.3 token/s on an 8GB RTX 4060

Based on host memory and PCIe bandwidth, FreeToken dynamically decides whether experts missing from the cache should be transferred to the GPU or computed directly on the CPU. The paper reports 1.3× to 2.1× higher decoding throughput than the strongest baseline across five consumer-grade systems, but the linked code repository is not currently publicly accessible.

2026-08-18

代理工具

Quipu Changes Agent Writes to Knowledge Graphs from Accept-Then-Clean to Validate-Before-Commit, with Rejected Records Replayable Across Versions

Quipu places data, trust labels, rules, and decisions on dual timelines, and checks candidate state before committing agent writes. In the authors’ injected-defect test, all six defects persisted without a gate, compared with zero under Quipu, though the evidence still comes mainly from a single deterministic evaluation designed by the system’s authors.

2026-08-18

推論系統

llama.cpp b10472 Fixes AMD APU Memory Detection, Preventing Strix Halo from Being Assigned Nonexistent Available Capacity

The new release stops overriding GPU memory information with the system’s `MemAvailable` value in Linux HIP builds and instead trusts `hipMemGetInfo`. The fix targets AMD APUs with configurable UMA carveouts and reduces the risk of the model loader overestimating capacity and failing later.

2026-08-18

訓練基礎設施

Rollplex Fits VLM Reinforcement Learning Prefix Computation into Rollout Idle Time, Accelerating Training by up to 1.30× on 32 H800 GPUs

Rollplex executes video encoding and prompt prefill for the reference model and training actor early, during autoregressive decoding, while preserving synchronous on-policy semantics. The system uses CUDA VMM to manage memory across stages and enables training and inference processes with different tensor-parallel configurations to share compatible weights.

2026-08-17

代理工具

ScienceFlow Adds Executable Workspaces to Agent Memory, Reaching a 70.22% Any-Medal Rate on 75 MLE-bench Tasks

ScienceFlow saves more than conversation summaries: it packages code, validation evidence, resource records, and workspace snapshots into recoverable stages. The official results report a 70.22% Any-Medal rate on the full MLE-bench, though cross-system comparisons remain affected by differences in models, hardware, and time budgets.

2026-08-17

推論系統

llama.cpp b10456 Fixes SYCL Quantization Transcoding Scheduling, Boosting Q4_0→FP32 Throughput on Arc 70 to 158.19 GB/s

The new release configures threads and work-groups according to quantization block size, preventing over- or undersubscription in the SYCL copy kernel. Microbenchmark throughput on Arc 70 rose from 20.21 to 158.19 GB/s, but this does not represent the speedup in token generation for a complete model.

2026-08-17