推論系統

推論 runtime、量化、 serving、KV cache、成本與效能。

TsuGO Reconstructs Search Trees from Reasoning Text; Gemini 3.1 Pro Scores Just 19% on Open-Ended Hard Problems

Using 600 verifiable Go life-and-death problems, TsuGO measures whether models allocate reasoning resources to the correct branches rather than merely counting correct answers and tokens. After four multiple-choice candidates are removed, both accuracy and search efficiency fall sharply across several reasoning models, leaving a clear gap with the neural-network-guided KataGo.

2026-08-15

最新模型

LongEarth-R1 Reasons Over Satellite Sequences of Up to 30 Frames, Outperforming Comparison Models Across All 12 Spatiotemporal Tasks

LongEarth-Bench organizes 117,000 remote-sensing images into approximately 120,000 long-sequence question-answer pairs, requiring models to track changes in landforms, anomalies, and missing states. LongEarth-R1, built on Qwen2.5-VL-7B, adds frame numbering, structured reasoning, and spatiotemporal rewards, but its data, weights, and code have not yet been released.

2026-08-15

推論系統

DARTree Builds Trees in Batches Before Pruning, Accelerating Qwen3-4B Decoding to as Much as 9.73× the Speed of Local AR

DARTree extends causal correction for diffusion draft models from a single path to multiple branches and replaces node-by-node heap search with batched expansion by depth. In tests on a single RTX 6000 Ada, it delivered higher average speedups across seven tasks than DFlash, DDTree, and Domino, but the code has not yet been released.

2026-08-14

推論系統

TEMPO Assigns MoE Experts Based on Measured Runtime, Cutting Qwen3-235B p99 Token Latency by About 15.6%

Rather than balancing only tokens or the number of active experts, TEMPO estimates both HBM weight-loading costs and GEMM tile padding. An SGLang prototype improves throughput by 4% to 6% on some Qwen3-235B workloads, but adds only overhead on the communication-bound DeepSeek-V3.

2026-08-14

推論系統

FlashDrive Combines Caching, Speculative Decoding, and W4A8 to Cut 10B Autonomous-Driving VLA Latency to 151 Milliseconds

FlashDrive tackles four bottlenecks—visual encoding, LLM prefill, reasoning tokens, and trajectory denoising—cutting Alpamayo 1.5-10B single-window latency on an RTX PRO 6000 from 717 to 151 milliseconds. The open-source implementation approaches real-time inference, but requires specialized fine-tuning, and its closed-loop evaluation covers only 100 simulated clips.

2026-08-14

最新模型

AlayaWorld v1.1 Switches to a Streaming 3D Point Cache, Raising Its Long-Horizon Video Consistency Score to 89.5

AlayaWorld has rewritten the spatial memory and conditioning pipeline of its interactive world model so that camera controls, historical frames, and generated content use aligned causal VAE representations. A new report claims the highest overall consistency score on WBench, but the updated weights and inference code have yet to be released.

2026-08-14

推論系統

GCache Allocates Diffusion Caching Based on End-to-End Error Impact, Cutting Wan2.1 LPIPS to 0.0316 at the Same Speed

GCache no longer decides whether to recompute based on local similarity between adjacent denoising steps. Instead, it searches for a cache-reuse schedule with the lowest impact on final image quality. On Wan2.1, it substantially reduces deviation while maintaining a 2.17× speedup, but policy search requires multiple H100 GPUs and the implementation has not yet been released.

2026-08-14

推論系統

vToken Reclaims KV Cache Through Token-Level Virtualization, Boosting Throughput by Up to 37% on a Single H100

vToken adds a token-to-physical-location mapping layer on top of PagedAttention blocks, compacting scattered KV gaps left after eviction. Its vLLM prototype reduces retained blocks by up to 72.3%, though testing remains limited to a single machine with one GPU and primarily 7B- and 8B-class models.

2026-08-14

推論系統

Trie Automata Precompute Token Masks, Delivering Up to 29× XGrammar’s Throughput for Finite-Choice Decoding in vLLM

A new method precomputes trie states and token masks for thousands to tens of thousands of valid strings, avoiding the cardinality bottleneck of general-purpose grammar compilation. The authors measured 219 requests per second at a batch size of 256, but have not yet released the implementation, and the figure also includes integration gains from bypassing the existing guided-decoding path.

2026-08-14

最新模型

64-Way Self-Consistency Voting Actually Reduces Per-Question Accuracy on Most Hard GPQA Problems

A preregistered study found that, on most GPQA Diamond questions, Qwen2.5-7B and Llama-3-8B become increasingly likely to lock onto the wrong answer as more samples are put to a vote. Overall average scores still rise slightly, showing how aggregate metrics can conceal per-question regressions.

2026-08-13

最新模型

LTX‑2.5 Updates Its Open Video Model With a Diffusion Decoder and Single-Pass Multi-Shot Generation

LTX‑2.5 releases trainable 22B weights and eight-step distilled weights, adding character, audio, and scene consistency across shots. The complete quick-start components total roughly 66 GiB, Diffusers support still requires installing from the main branch, and the model is subject to a community license with a revenue threshold.

2026-08-13

最新模型

Qwen3.8 Makes 2.4T-Parameter Max-Class Weights Available for the First Time, Activating 95B per Token

Alibaba’s Qwen team has released Qwen3.8-2.4T-A95B, featuring 512 experts and a hybrid of linear and standard attention, with support for contexts of up to approximately 1.01 million tokens. The model offers adjustable reasoning depth, but accepts text only and cannot disable thinking mode, making its practical deployment requirements far higher than those of typical open-weight models.

2026-08-13