Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
推論系統
TypeSafe AI’s Jev produces only predefined booleans, choices, and scores, seeking to turn model inference into low-latency branches that can be embedded directly in software. Type-correct outputs do not guarantee correct decisions, and the company’s speed and cost claims still rest on in-house workflows and undisclosed models.
AI 安全
New research finds that malicious plans phrased in benign language and inserted into context can be absorbed, rewritten, and executed by models, with even the strongest monitors missing at least a quarter of attacks. Showing monitors the original injected content or increasing the reasoning budget does not reliably improve detection, suggesting that agents cannot establish their safety solely through their output chain of thought.
代理工具
The new open-source Reef collects traceable interactions through OpenAI- and Anthropic-compatible inference endpoints, then asynchronously updates model weights or the agent harness. It adds the control plane needed for hot updates and version rollbacks, but training-data poisoning, evaluator bias, and production stability remain to be validated.
最新模型
Google has launched two native-audio models. Extended Thinking can perform background reasoning and asynchronous tool calls while continuously streaming audio. The new protocol improves concurrency for long-running voice agents, but also requires developers to rewrite session state machines and interruption handling.
最新模型
ShadowPEFT models task adaptation as a small trainable model running alongside a frozen backbone and is now integrated with PEFT’s standard configuration, saving, and loading interfaces. Early experiments outperform LoRA and DoRA at comparable parameter counts, but additional memory usage, dual KV caches, and the lack of a stable release remain deployment trade-offs.
代理工具
Cline has released an open-source desktop beta that lets multiple agents run concurrently in isolated sessions and supports unattended tasks through cron scheduling. However, the model results highlighted by the company come from an existing CLI harness, not an independent evaluation of the desktop app.
推論系統
Portable Computer can now run models, tools, and sandboxes on Windows RTX PCs with at least 24GB of VRAM, with most local work not counting against cloud quotas. However, the system can still escalate to cloud models with user authorization, while connectors and external actions mean it should not be treated as a fully offline environment.
最新模型
The new architecture repeatedly applies a three-layer shared block, allowing a router to decide per token whether to exit early or continue latent reasoning. In experiments with roughly 1 billion parameters, the paper reports better latency and accuracy than a fixed-depth looped Transformer, though the approach has yet to be validated on frontier models or mainstream inference engines.
代理工具
Google Research has unveiled Stellar Colosseum, which manages long-horizon mathematical research through strategy exploration, counterexample attacks, dependency graphs, and hierarchical aggregation. The system scored 71% on the 300-problem TCS-Bench, but the result relies on extensive inference and automated grading and should not be treated as evidence of a model’s independent proving ability.
AI infrastructure
SemiAnalysis tested the Vera Rubin NVL72 using pre-release software and reported up to roughly seven times the token throughput per megawatt of Blackwell. The results highlight the role of HBM4 and NVLink 6 in long-context MoE inference, but the headline 67-fold price-performance gain incorporates rental pricing and assumptions about specific operating points.
代理工具
Copilot Auto now lets users adjust the weighting of cost, quality, and latency across three tiers—Efficiency, Balance, and Intelligence—but all three share the same model pool. Actual charges still depend on the model selected by the router and the number of tokens used; tier names are neither price caps nor guarantees of a specific model.
最新模型
The Shanghai AI Laboratory has released Atria Dawn Preview, along with FP8 weights and inference recipes, positioning it as a full-loop agent capable of searching, coding, and running experiments. The organization claims the best reported results on five of 16 benchmarks, though model scale, test-time compute, and its in-house evaluation setup still limit comparability.