代理工具

Agent、MCP、coding agent、工作流自動化與評測。

IOL-AI Tests Reasoning on Unreleased Linguistics Olympiad Problems: A 14B Model More Than Doubles Its Baseline Score Through Its Decoding Pipeline

The IOL-AI Challenge requires systems to infer the rules of an unfamiliar language from a small number of examples before completing translation, transcription, and correction tasks. Under a single-T4 constraint, the best submission raised the same model’s baseline GM score from 9.40 to 19.79, but still lagged well behind closed models that reached the gold-medal threshold.

2026-08-19

AI 安全

PACE Binds AI Agent Transaction Approvals to Onchain Bytes, Cutting the Unsafe Execution Rate From 0.80 to Zero in Sandbox Tests

PACE does not ask language models to determine whether transactions are safe. Instead, it uses deterministic rules to check transaction intent, simulation results, and actual calldata. A smart account revalidates the signed policy decision record before execution, though the zero-failure result so far comes only from a simplified DeFi sandbox.

2026-08-19

硬體

PTXBench Tests LLMs Writing H100/B200-Specific PTX Directly; No Model Reliably Matches State-of-the-Art Kernel Libraries

PTXBench checks whether generated kernels are correct, whether specified instructions actually execute, and how their performance compares with cuBLAS, cuDNN, and FlashInfer. The results show that generating instructions for new architectures does not necessarily translate into effective acceleration, with B200 attention kernels proving especially difficult.

2026-08-19

代理工具

StagedWorkspace Uses Content Hashes to Synchronize Agents’ Parsed Views and Native Files, Boosting OfficeQA Pass@1 by 8.3 to 12.1 Points

The new system binds a document agent’s search results, native files, diff views, and final submissions to explicit versions, preventing the agent from reading stale parsed content while editing a newer file. Ablations within a fixed framework show that providing both parsed and native views outperforms either view alone, though the full codebase and cross-framework reproductions are still needed.

2026-08-19

AI 安全

Medical LLMs Give Self-Care Advice Before Patients Respond; a Short Instruction Cuts the Rate from 9/12 to 0/12

A new study moves evaluation upstream to the initial-contact stage, before patients have clearly described their problems. Adding a “clarify first, advise later” system instruction improved response sequencing and handoff summaries, but still did not reliably elicit decisive medical history.

2026-08-19

TokEval Screens Tokenizers Using 18 Intrinsic Metrics, With Information-Theoretic Metrics Correlating With Language Modeling Performance by Up to 0.80

TokEval incorporates numeric place value, program AST boundaries, UTF-8 integrity, and cross-lingual cost into tokenizer evaluation. Controlled experiments show that some metrics can predict specific downstream capabilities, although the results are currently limited to a single 1.27B-parameter architecture.

2026-08-19

代理工具

Agent Lightning v1.0 Brings Real-World Agent Frameworks Directly Into Reinforcement Learning, Raising Qwen3.5-9B to 56.4% on SWE-bench Verified

Microsoft rewrote Agent Lightning to capture complete interactions from existing agent frameworks through a model API proxy layer, eliminating the need to move tool loops into the training engine. Using 6,000 training examples, the team reports that Qwen3.5-9B improved from 41.8% to 56.4% on SWE-bench Verified.

2026-08-19

代理工具

Self-Improving Agents Face Random Task Orders, Reversing the Average Gain From +1.5% to −4.5%

Salesforce reevaluated two text-based memory methods and found that adding a self-improvement loop increased run-to-run variance in 71% of configurations. Once the fixed task order—which implicitly formed an easy-to-hard curriculum—was shuffled, ReasoningBank not only failed to improve the agent but reduced its success rate.

2026-08-19

代理工具

AeroCopilotBench Turns Flight Procedures Into Executable State Machines, Yet the Best Agent’s Safe Pass Rate Is Still Only 72.6%

The new benchmark uses 73 abnormal cockpit tasks to test whether agents can observe state, operate systems, and comply with hard safety constraints throughout—not merely answer aviation knowledge questions. GPT-5.6 Sol achieved the highest success rate, but similar knowledge scores did not correspond to similar task success rates, showing that static question answering cannot replace interactive safety testing.

2026-08-19

代理工具

Reconstruction Hides Papers and Provides Only Older References; Multi-Model Review Raises Research-Idea Match Rate from 14.6% to 35.6%

A new benchmark asks models to reconstruct a hidden paper’s core research idea using only references that existed before its publication. Cross-review by four models and Swiss-system selection improve the match rate, but the candidate count, judge pool, and compute budget are not aligned with the single-model setting.

2026-08-19

代理工具

BaT Schedules Reinforcement Learning Around Agent Failure Stages, Raising a 9B Policy’s AutoMedBench-Lite Score From 19.9 to 53.4

Benchmark-as-Teacher turns stage-level evaluation scores directly into the curriculum for the next training round, while using isolated synthetic data to prevent test answers from leaking into updates. The paper reports that its 9B policy substantially outperforms standard GRPO, but the full training dataset has not yet been released, and cross-system rankings mix different execution frameworks.

2026-08-19

FabriMAE 以視覺注意力熵判斷機器人動作可靠度，十分支選擇令成功率升至 86.8%

FabriMAE 從 VLA 模型內部的視覺注意力熵產生可靠度分數，不需另訓練驗證器或重複執行機器人軌跡。PI0.5 在 LIBERO-Plus 的成功率增加 1.1 個百分點，但目前證據仍限於模擬環境，論文連結的程式庫亦尚未公開存取。

2026-08-18