Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
推論系統
Yutori’s Navigator calls a vision-language model dozens of times during each browser task, with historical context growing to roughly 100K tokens. Together AI handles this workload using vLLM prefix caching and separate latency targets, although the claimed 2× speedup and 4–5× lower cost remain vendor-reported comparison results.
代理工具
Nous Research has rolled keyless web search, persistent memory for scheduled tasks, and execution-stall safeguards into Hermes Agent 0.20.5. Although labeled a patch release, it merges roughly 323 PRs; full migration notes will not arrive until 0.21.0.
推論系統
Hugging Face now lets MiniMax H3, MiniMax Music 3, and Wan-Animate-2 plug in directly as composable blocks, without traditional DiffusionPipeline wrappers. The release also adds limited tensor parallelism, low-bit quantization backends, and security fixes, but removes Flax and changes LoRA outputs.
推薦系統
NVIDIA has assembled HSTU, dynamic embeddings, and Semantic ID-based generative recommendation into an executable PyTorch reference stack. It rewrites the caching and decoding paths for recommendation workloads characterized by long histories, generation of only a few tokens, and extremely large beam widths, though the results are currently limited to NVIDIA hardware and synthetic configurations.
推論系統
Liquid AI has released DSpark draft checkpoints with about 300 million parameters for three LFM2.5 models and integrated them with llama.cpp and SGLang. Official tests show that acceleration varies significantly by model, data distribution, and hardware, so peak results should not be treated as representative of gains across all workloads.
代理工具
AI4AI-Bench evaluates hyperparameter tuning separately from genuine changes to learning methods, requiring coding agents to submit training code that can be rerun from scratch. Across 290 trials involving six systems, more than 40% underperformed the original algorithms, while increased reasoning effort mainly made agents more willing to modify core training mechanisms.
AI 基礎設施
DSX MaxLPS uses real-time telemetry to reallocate unused power. NVIDIA found that the provisioned power for a GB200 NVL72 rack could be reduced from 125 kW to 90 kW while maintaining throughput. The solution also combines workload-specific power profiles with 45°C liquid cooling, but its core Dynamic Power Software remains in Developer Preview.
推論系統
FormalTCS extracts research-grade problems from top theoretical computer science papers published over the past two years, measuring models stage by stage as they turn natural-language claims into machine-verifiable proofs. The best model achieves a Pass@8 of 28.6 when proving already-formalized theorems, but translating natural language into correct Lean theorem statements remains the primary bottleneck.
推論系統
Daedalus-150M is designed specifically for single-user, 4-bit CPU inference, allowing most network layers to avoid repeatedly reading an ever-growing KV cache. Preliminary results indicate that its speed advantage increases with context length, but the model is limited to 2K context, and its performance figures still await independent reproduction across different hardware.
代理工具
NVIDIA has updated its open-source agent execution stack, focusing on state drift across vLLM, llama.cpp, Ollama, and the model router. The new release also tightens sandbox lifecycle management, image provenance, and failure recovery, though it remains a patch release in a rapidly evolving project.
Task-CoEvolve dynamically selects validation tasks based on historical disagreement among candidate harnesses, then estimates overall performance using sampling probabilities. On Terminal-Bench 2.1, it approaches full-set search performance, but the paper uses the same tasks for search and evaluation, and the code has not yet been released.
推論系統
A new method lets models adapt directly to KV eviction strategies such as H2O during fine-tuning, raising accuracy on some 128K tasks from roughly 20%–64% to over 90%. However, it does not consistently lead on natural question-answering benchmarks, and the open-source implementation remains positioned as a research tool.