代理工具

Agent、MCP、coding agent、工作流自動化與評測。

代理工具

FM-Bench Has Agents Run a Football Club for 20 Years; None of 15 Models Could Infer Market Prices From Rejected Bids

Using 26 tools and roughly 340 to 400 decision points, FM-Bench tests whether agents can handle delayed rewards, hidden information, and competitors’ reactions. All 15 models completed the 20-year campaign, but rankings stabilized only in the later stages, while self-managed memory exhibited two failure modes: endless accumulation or repeated overwriting.

最新模型

Recurrent Inference Improves Multi-Tool Composition, Raising the 1B Llama’s Average BFCL Score from 21.4 to 32.9

Researchers repeatedly update hidden states through the same set of Transformer layers before generating each token, comparing recurrent and conventional models using identical data and LoRA configurations. Gains are concentrated in parallel calls and cross-call dependencies, while API-Bank, which primarily involves single API calls, shows no consistent advantage.

AI 安全

VLA Monitors Hidden-State Communication Between Agents, Achieving 0.993 AUROC in Same-Model Auction Tests

A new study links continuous vectors exchanged between agents—but omitted from conversation logs—to their subsequent public actions as a single event, then applies three layers of checks: anomaly detection, counterfactual impact analysis, and sparse-feature inspection. The average AUROC of 0.993 combines text-based and latent-collusion samples, and the attacks were predesigned by the researchers, so the result cannot yet be considered a general capability for detecting emergent conspiracies.

代理工具

LLM Judges Decide First, Then Search or Abstain, Using Calibrated Thresholds to Control the Error Rate

A new framework uses predictive entropy to determine whether an LLM judge should accept a verdict directly, search for web evidence, or abstain, while calibrating two thresholds with a Clopper–Pearson upper bound. All 32 open-domain question-answering experiments maintained the specified error rate, but the guarantee depends on i.i.d. calibration data and reliable labels.