推論系統

推論 runtime、量化、 serving、KV cache、成本與效能。

KoboldCpp 1.120 Adds Direct I/O Loading Mode, Allows mmap and mlock to Be Enabled Together

The new release adds `--usedirectio` for large GGUF weights and allows the relevant memory pages to be locked when a model is mapped with mmap. It also supports Qwen3.8-Flash-Next, Ling-3.0-flash, and custom JavaScript tools, but the project has not published loading-speed or memory benchmarks.

2026-08-30

硬體

Sopro V2 Turbo Releases a 120M-Parameter Speech Model with Two-Step Acoustic Solving for Streaming Synthesis on CPUs

Sopro V2 Turbo distills a 500M-parameter teacher into a 120M-parameter model, then uses reflow to reduce acoustic solving from 32 steps to two. The model can produce its first audio chunk in about 300 milliseconds on an M3 CPU and supports in-browser inference, but it currently covers only four European languages and does not include training code.

2026-08-30

代理工具

SKILL.state Discards Agent Execution History, Cutting Token Use for 100-Step Tasks to One-Sixteenth With Structured State

Researchers from Google and Purdue University designed an agent that reads only a fixed skill specification, its current state, and the latest observation at each step, immediately discarding its reasoning history after validating state updates. In a 100-step warehouse test, it achieved 0.94 accuracy using 65,408 tokens, but its effectiveness depends on a predefined state structure preserving information needed later.

2026-08-30

最新模型

Hy4 Preview Opens 770B MoE Weights, Targeting Long-Context Agents with Sparse Attention and Built-In MTP

Tencent has released Hy4 Preview’s Apache 2.0-licensed weights, an FP8 version, and deployment recipes for vLLM and SGLang. The model activates roughly 49B of its 770B parameters per token. It supports a 1-million-token context window, though evidence for its quality and self-optimization capabilities still comes primarily from Tencent’s internal evaluations.

2026-08-28

推論系統

llama.cpp Reworks Adreno Matrix Multiplication Routing, Speeding Up Prompt Prefill by 9% to 25% Across Two Chip Generations

llama.cpp b10687 avoids inefficient matrix multiplication paths selected by the OpenCL compiler on Snapdragon X2 and older A7X GPUs. Maintainers measured prompt-prefill speedups of roughly 20% to 25% for gpt-oss-20b and 9% for Gemma 3n, while per-token decoding saw no improvement.

2026-08-30

最新模型

FastH3 Compresses MiniMax H3 to Four DiT Calls, Generating 15 Seconds of Video in 13 Seconds on Eight B200 GPUs

FastVideo combines DMD2 distillation with 90% sparse attention to reduce the base model’s 49 DiT calls to four, while releasing the full weights and LoRA. Its real-time generation figures depend on eight B200 GPUs, specialized VSA kernels, and fully warmed-up tests; a single GPU and mainstream consumer GPUs have yet to achieve comparable speeds.

2026-08-30

AI 安全

NemoClaw 0.0.116 Persistently Binds Sandbox Identity, Tightening Recovery, Takeover, and Deletion Permissions

After strengthening local inference and state-drift validation in 0.0.113, NVIDIA has further bound sandbox, inference service, and lifecycle permissions to persistently verifiable identities, preventing retry or recovery operations from taking over an environment with the same name but a different identity.

2026-08-30

AI 研究

CritICL Turns Small-Model Failure Modes Into a Prompt Library, Bringing Single-Generation Performance Close to Multi-Sample Test-Time Compute

CritICL first catalogs small-model errors, failure types, and critiques offline, then adds relevant examples to a large model’s prompt instead of repeatedly sampling for every question. The authors report slightly higher accuracy than self-consistency on Qwen2.5 math benchmarks, but the cost of building the library offline and the unreleased code limit reproducibility.

2026-08-30

代理工具

PILOT Lets Supervisory Agents Reroute Mid-Task and Write Execution Experience Back into Persistent Skills in Real Time

PILOT connects a supervisor and a worker agent through a bidirectional channel, allowing the former to provide guidance or terminate an erroneous path before a long-running task is complete. The authors report improved accuracy and token efficiency across three agent benchmarks, but its “self-improvement” updates only the harness, skills, and memory—not the model parameters.

2026-08-30

最新模型

Puro-2B Publishes RTX 5090 Pretraining Recipe, With a Reported Cost of $6,891 for 1.4T Tokens

Puro-2B integrates a two-stage data curriculum, blockwise FP8, and MuonH optimization on a consumer-grade GPU cluster. The authors report that a shortened $4,370 training run already surpasses Qwen2-1.5B in average score, although the cost accounts only for accelerator rental-equivalent value.

2026-08-29

最新模型

TTPO Routes Positive and Negative Trajectories by Majority Vote, Enabling Models to Update at Test Time Without Ground-Truth Answers

Rather than treating the majority-vote answer as ground truth, TTPO distills trajectories that agree with the consensus and uses grouped reinforcement learning to penalize high-confidence disagreements. The authors report that Qwen3-1.7B’s Avg@12 across three mathematics competitions rises from 38.0 to 45.2, though this remains a transductive setting in which parameters are updated directly on the test problems.

2026-08-29

推論系統

Prefix Sliding Preserves the Prompt Prefix and Recent Reasoning, Turning KV Memory for Ultra-Long Thinking into a Fixed Budget

During generation, Prefix Sliding evicts intermediate reasoning tokens while retaining the system prompt, task prefix, and the most recent several thousand tokens. It can be applied without retraining. The research reports that overall thinking time for long reasoning can be reduced by roughly threefold, but the current implementation depends on forked versions of vLLM and FlashAttention, with limited benefits for short outputs and tasks that require revisiting early details.

2026-08-29