代理工具

Agent、MCP、coding agent、工作流自動化與評測。

ORCA-bench Reconstructs Six Days of Telemetry, With Frontier Agents Achieving Just 25.3% on Medium-Difficulty Root Cause Analysis

Using runnable microservices, 50 GB of telemetry data, and 1,079 incidents, ORCA-bench tests whether coding agents can identify root causes from ambiguous incident reports. Even with full source-code access, the best agent correctly answered only one-quarter of the medium-difficulty tasks designed to resemble real-world inputs.

2026-08-02

A Batch of Older Mistral Models Will Be Retired on July 31, Requiring Regression Testing for Coding Agents and Structured Outputs

Mistral’s summer model cleanup is entering its final batch, affecting older identifiers for Devstral, Magistral, Mistral Small, and Open Mistral Nemo. The officially recommended replacements are not snapshots of the same architectures, so migration requires more than renaming models: teams must revalidate tool calling, output schemas, context windows, and costs.

2026-08-02

基準與基礎設施

MLPerf Endpoints v0.7 Launches, Adding Continuous Submissions and a Comparable Results Pipeline for Agentic Inference

MLCommons has released Endpoints v0.7, transforming inference testing from fixed-batch submissions into endpoint benchmarking infrastructure with automated review and continuous updates. The upcoming multi-turn agentic workloads will measure both per-user progress and aggregate throughput, rather than counting only tokens/s for isolated requests.

2026-08-02

DeepSeek-V4-Flash 0731 Redoes Only Post-Training, Raising Its Official Terminal-Bench 2.1 Score to 82.7

DeepSeek has moved the newly post-trained V4-Flash into public API beta without changing its architecture, 284B total parameters, or 13B active parameters. The new version natively supports the Responses API and targets coding agents and tool use, but key benchmarks rely on the unreleased DeepSeek Harness.

2026-08-02

RAG 與檢索系統

GLM-RAG Feeds Knowledge Graph Text into a Graph Transformer, Outperforming GNN Baselines on Cross-Domain Multi-Hop Retrieval

GLM-RAG allows nodes, relations, and questions to interact jointly from the first attention layer instead of relying solely on message passing based on graph distance. It performs better on multi-hop evaluations in unseen domains, but standard vector RAG still leads on single-hop tasks, and the code and model have not yet been released.

2026-08-01

AI 安全

DFOT Quantifies LLM Over-Trust in Derived Sensor Values; Cross-Modal Distillation Improves Correction Rates by Up to 6.69 Points

A new framework breaks down “model over-trust in derived values” into metrics covering conflict acceptance, contextual misdirection, error correction, and utility harm. In a PPG-based rhythm interpretation case study, a reliability model that uses ECG during training but requires only PPG at deployment improves four endpoints, although the data and weights were not released with the code.

2026-08-01

DualG-MRAG Splits a Multimodal Knowledge Graph into Two Layers, Raising MMQA R@5 to 61.9%

DualG-MRAG uses a Macro Graph to find cross-document reasoning paths, then a Micro Graph to verify local image and text evidence, preventing fine-grained nodes from rapidly inflating the knowledge graph. In MMQA, the study raises R@5 from 42.1% for the strongest graph-based baseline to 61.9%, but the code and index construction costs have not yet been disclosed.

2026-08-01

代理工具

GRSD Assigns Rewards by Comparing an Agent’s Own Success and Failure Trajectories; ALFWorld Unseen Task Success Rate Reaches 83.6%

Group-Reflective Self-Distillation no longer extracts fixed skills from an external teacher. Instead, it compares successful and failed rollouts from the same policy and converts their differences into turn-level training signals. It delivers overall improvements across three agent environments, but increases training time by 17% over GRPO and still requires an external model to score reflection text.

2026-08-01

GitHub Repo

GCC Rejects Copyright-Significant LLM-Derived Content, While Maintainers May Make Exceptions for Generated Tests

The GCC Steering Committee has approved an AI contribution policy that generally rejects substantial code and text containing or derived from LLM output. Models may still be used for debugging, analysis, and review, and maintainers may accept generated tests, but their output cannot be incorporated directly into ordinary contributions.

2026-08-01

代理工具

Change2Task Ports Historical PRs to Modern Code, With 900 Agent Tasks Passing Three-Stage Validation

Change2Task extracts development intent from merged PRs, then reconstructs executable tasks on a modern version of the same repository. The study generated 900 task sets from 1,130 candidate changes, although most cases still required another coding agent to complete the reconstruction.

2026-08-01

代理工具

MemHarness Rewrites Agent Memory Before Acting, Raising 7B Model’s ALFWorld Success Rate to 85.2%

Instead of replaying retrieved past experiences verbatim, MemHarness has the same policy model critique and reorganize them based on the current state before deciding how to act. After end-to-end training with GRPO, it achieved success rates of 85.2% on ALFWorld and 75.6% on WebShop, though the code and model weights have not yet been released.

2026-08-01

LLMOps

Grafana Agent Observability Reaches General Availability, Connecting Online Evaluation and Agent CI Through Open-Source SDKs

Grafana has made its internally used agent observability platform generally available and released the open-source agento11y SDK, which supports five languages and multiple agent frameworks. The platform brings conversations, tool execution, costs, OTel traces, online evaluation, and offline experiments into a unified regression workflow, although the complete product remains tied to Grafana Cloud.

2026-08-01