Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
推論系統
ReCo dynamically compresses the KV cache based on process rewards for each reasoning step, while suppressing repetitive reflection and enabling early stopping. In the authors’ experiments across three reasoning models and six benchmarks, generated tokens fell by 37% to 65%, while latency improved by 2.08× to 2.35×.
AI 基礎設施
Microsoft Azure research shows that agents repeatedly switch among model inference, tools, and orchestrators, putting the CPU back on the critical path while GPUs often sit idle. The Agora prototype can reclaim compute based on workflow patterns, but incorrectly reducing GPU allocations for parallel agents can worsen tail latency by 16×.
代理工具
ABSeeker breaks down a search agent’s final success or failure into step-level rewards, preserving useful queries from failed trajectories while downweighting erroneous steps in successful ones. The team has released the 4B model weights and the inference and training pipelines, though its best results still rely on additional context management and an LLM judge.
代理工具
A new study re-audits latent communication in multi-agent systems using mismatched, zeroed, and random KV caches, finding that the large performance drop caused by removing the cache does not necessarily prove that the receiver used specific information from the sender. On some standard benchmarks, zeroing the cache reduced accuracy by 14.7 points, while replacing it with a cache from another problem cost only 0.4 points.
AI 安全
A new study uses paired prompts to compare whether models leave detectable evidence in their chains of thought when influenced by explicit instructions versus implicit cues. Across seven reasoning models, answers still changed under some implicit settings, while monitor detection rates were as much as 46 percentage points lower than in explicit settings.
最新模型
A new study tests instruction-tuned models using speech transcription and QWERTY keyboard perturbations, finding that structural rewrites introduced by speech consistently reduce answer performance. Increasing the reasoning budget almost fully repairs degradation from the keyboard channel but cannot similarly rescue spoken input.
推論系統
The new method maps a KV cache generated by a smaller model to a larger model in the same family, avoiding reprocessing the entire prompt when switching models during long conversations. Only four of six model pairs retained 73% to 98% of the original accuracy, showing that tensor compatibility does not guarantee semantic compatibility.
最新模型
Rather than adding multiple full parameter sets, ParVL reuses the same ViT and LLM weights and distinguishes compute paths with branch-specific KV prefixes. The research shows that different tasks require different ratios of vision and language branches, so a fixed configuration may waste inference compute.
AI 安全
AgentS4D tested 20 agent framework–model combinations using 328 risk-injection cases. Of 6,560 runs, 68% triggered predefined danger signals. More notably, 4,344 runs both completed the task and were deemed unsafe, showing that relying solely on success rates systematically underreports agent risks.
AI 安全
Rather than generating a complete safety rationale for every review, LatentGuard uses staged training to compress rationales into a small number of latent states and output decisions directly. An independent audit decoder can reconstruct brief explanations on demand, but evidence for its interpretability and efficiency currently comes only from the authors’ experiments.
推論系統
Oilbird adds semantic indexing to training-free speculative decoding, recovering existing continuations missed by exact string matching when a few parameters or tokens differ. The researchers report a 24% to 29% increase in average acceptance length across three drafters, but have not yet released independently reproducible code.
代理工具
LiveEvalBench uses three types of agents—build, code, and browser interaction—to gather evidence, then generates checks based on each project’s actual functionality. The framework and dataset are public, but scoring still relies on LLM judges, and its cost, reproducibility, and judge bias require further quantification.