Archive
Agent、MCP、coding agent、工作流自動化與評測。
代理工具
An Amazon research team transformed existing tool-use benchmarks into fault environments in which agents can be instructed to retry, switch tools, or stop. After structured recovery information was added to Qwen3-4B, its pass rate on disrupted retail tasks increased by as much as 16.8 percentage points, but the system has yet to truly validate correct abstention on unsolvable tasks.
生成式影音與 3D
StateFlow stores object geometry, poses, semantic states, and camera settings as an editable 3D world, leaving the video model to handle visual refinement. In a 30-person study, its video previews received an overall rating of 4.5/5, though the system relies on several closed models and its code has not yet been released.
代理工具
In a strong-to-weak test-time transfer experiment, stronger models wrote routing, format-checking, and deterministic-solving programs for a fixed smaller model without updating its weights. The best harness corrected about 83% of baseline errors, but performance depended heavily on theory-of-mind benchmarks whose patterns could be compiled into rules.
代理工具
A LoCoMo experiment found that general-purpose summaries preserve people and events but selectively discard dates, times, and durations. After an explicit instruction to preserve temporal anchors verbatim, judged accuracy on temporal questions rose from 15.6% to 47.0%.
代理工具
CTBench requires agents to query interfaces modeled on real-world equipment and evaluates root-cause analysis and path reconstruction against expert-defined evidence steps. Although the best configuration raised path-reconstruction accuracy to 87.96%, root-cause analysis accuracy remained only 47.62%, with even less critical evidence retrieved.
代理工具
IBM’s VAKRA puts APIs, document retrieval, and natural-language policies into a single executable trajectory. Even the strongest model achieves only 50% to 70.4% full correctness across different API interfaces. When policies make questions unanswerable, some models’ success rates fall as low as 2.4%.
代理工具
Using paired trajectories with and without skills, teams including Microsoft Research identified 125 functional failures and 182 cases of efficiency regression. The problem was usually not that the skill was irrelevant, but that the agent treated generic examples, validation checklists, or environment assumptions as hard task requirements.
代理工具
DashArena evaluates model-generated interactive data dashboards across 234 open-ended tasks and uses a browser to replay the interaction workflows written by the models themselves. Adding interaction evidence raised agreement between an 8B vision judge and human preferences from 71.7% to 79.8%, yet even the best model achieved a full replay rate of only 73.3%.
AI 安全
IO Factory connects narrative planning, agent posting, platform exposure, audience-state updates, and baseline comparisons into a traceable simulation lifecycle. The system has completed runs with 100,000 simulated civilians and 10,000 operator agents, but all influence measurements derive from manually configured rules and LLM judges.
代理工具
ReTree stores a search agent’s summaries, evidence sources, and revision history as a dependency tree. When new information overturns an earlier premise, the system returns to the node that introduced the error and deletes the affected branches. Results across four search benchmarks show that it both shortens per-step context and improves answers, though hard pruning may discard information that is not actually affected by the conflict.
代理工具
Rather than updating the small model’s weights, SKILLER has a stronger model repeatedly revise text-based skills using execution traces and verifier feedback. Qwen3.5-9B’s three-run average on SWE-Skills-Bench rose from 26.2% without skills to 82.8%, although skill generation still relies on GPT-5.4 and successful reference trajectories.
最新模型
Meta has released the Muse Glimmer 30B weights, vision encoder, and DFlash draft model. The 17GB version can run text, image, and speculative decoding workloads simultaneously within 24GB of VRAM. Meta measured an average of 233.4 tokens/s on an RTX 5090, but safety evaluations and deployment details show that the model still requires external safeguards and a newer runtime.