All News

1537 technical news articles

Model Main Effects Fall Below 3% Across Three Agent Benchmarks, as DDR Recasts Leaderboards as a Deployment Reliability Problem

A new study uses generalizability theory to decompose results from TheAgentCompany, τ²-bench, and AppWorld, finding that agent–task interactions far outweigh average differences between agents. The authors propose the DDR reporting framework, which requires model-selection decisions to disclose task difficulty, holdout validation, cost, and required sample size.

2026-08-14

代理工具

LLM Agents Send HPC Workloads to Heterogeneous Clusters, with Descriptive Hardware Data Raising the Success Rate from 48% to 87%

An LLNL team enabled agents to explore resources managed by Flux, Slurm, Kubernetes, and other systems, then translate high-level intent into verifiable job specifications. Experiments across six cloud clusters show that architecture, network, and memory metadata mainly rule out machines incapable of running a workload, rather than serving as a reliable performance optimizer.

2026-08-13

代理工具

GSR Compiles Scoring Rules into Typed Graphs, Improving Exact-Score Agreement for LLM Judges by Up to 6.75 Points

Graph-Structured Rubrics compile natural-language rubrics into typed judgment, transformation, aggregation, and gating nodes before seeing the response to be evaluated. Experiments with GPT‑OSS‑120B show that GSR outperforms flat Prometheus-style scoring, although no implementation has been released and the gains vary substantially by dataset.

2026-08-13

最新模型

64-Way Self-Consistency Voting Actually Reduces Per-Question Accuracy on Most Hard GPQA Problems

A preregistered study found that, on most GPQA Diamond questions, Qwen2.5-7B and Llama-3-8B become increasingly likely to lock onto the wrong answer as more samples are put to a vote. Overall average scores still rise slightly, showing how aggregate metrics can conceal per-question regressions.

2026-08-13

最新模型

MiniMax H3 Releases 33B Base Weights with Native Synchronized Video and Stereo Audio Generation

MiniMax H3 uses a single Transformer to jointly predict video and audio latents, supporting generation from text, first and last frames, and multimodal references. The open release can generate 768p audiovisual content locally, but the official 2K workflow still relies on two proprietary cloud modules.

2026-08-13

代理工具

Harness-IF Separates “What Models Would Do Anyway” From Genuine Instruction Following, Finding Gaps of Up to 7.4 Points Across 12 Coding Agents

Harness-IF distributes rules across system prompts, project files, user instructions, tool descriptions, and skill descriptions, then scores each rule using evidence from actual execution. All 12 frontier models became less accurate when rules conflicted with their default tendencies.

2026-08-13

AI 安全

VICBench Traces 100 Vulnerabilities to Their Initial Introducing Commits; Existing Automated Methods Reach Only 40.1% F1

Using independent manual and agent-based tracing, VICBench labels the vulnerability-introducing commits for 100 CVEs across Python, Java, and C++ projects. Cross-file moves and refactoring cause extensive misclassification by V-SZZ and LLM4SZZ, while also exposing reliability problems in affected-version data.

2026-08-13

最新模型

LTX‑2.5 Updates Its Open Video Model With a Diffusion Decoder and Single-Pass Multi-Shot Generation

LTX‑2.5 releases trainable 22B weights and eight-step distilled weights, adding character, audio, and scene consistency across shots. The complete quick-start components total roughly 66 GiB, Diffusers support still requires installing from the main branch, and the model is subject to a community license with a revenue threshold.

2026-08-13

最新模型

Qwen3.8 Makes 2.4T-Parameter Max-Class Weights Available for the First Time, Activating 95B per Token

Alibaba’s Qwen team has released Qwen3.8-2.4T-A95B, featuring 512 experts and a hybrid of linear and standard attention, with support for contexts of up to approximately 1.01 million tokens. The model offers adjustable reasoning depth, but accepts text only and cannot disable thinking mode, making its practical deployment requirements far higher than those of typical open-weight models.

2026-08-13

最新模型

Rankings Reverse for the Same Models as Token Limits Change, Four-Model Test Records 56,476 Inferences

A new study increased the generation limit for four open-weight models from 64 to 4,096 tokens and found that the leading model changed on all three reasoning benchmarks. Even after truncated outputs were excluded, 3% to 19% of questions changed from correct to incorrect as the budget increased, showing that a single token setting is insufficient for deployment decisions.

2026-08-13

代理工具

Bench2Robust Injects Nine Types of Tool Failures; Performance Regresses in 69 of 70 Agent Evaluations

An Amazon research team transformed existing tool-use benchmarks into fault environments in which agents can be instructed to retry, switch tools, or stop. After structured recovery information was added to Qwen3-4B, its pass rate on disrupted retail tasks increased by as much as 16.8 percentage points, but the system has yet to truly validate correct abstention on unsolvable tasks.

2026-08-13

AI 安全

New Claude Models Will Embed Text Watermarks During Generation, Including via the API and Claude Code

Anthropic will add model-level text watermarks to Claude models released after August 2, 2026, while embedding signed C2PA provenance data in supported files. The watermark can only indicate that content may have been processed by Claude; it cannot prove that a work was originally created by AI. No public detection tool or empirical accuracy results have yet been released.

2026-08-13