推論系統

推論 runtime、量化、 serving、KV cache、成本與效能。

SenseNova-U1.5 Publishes Report on Its Native Unified Vision Architecture, but the Full Training Pipeline Has Yet to Be Released

SenseNova-U1.5 uses a Mixture-of-Transformers (MoT) architecture that does not rely on a separate vision encoder or VAE, enabling a single model to handle understanding, reasoning, generation, and editing. The technical report added on September 11 details its training methods and evaluations. Existing weights and inference code are available, but the SFT, RL, and multi-expert distillation code has not yet been fully released.

2026-08-24

推論系統

H3 Lightning Cuts MiniMax H3 Video Generation to Four Steps, but Its 12.2× Speedup Depends on Unreleased Weights

RunningHub has published a multi-GPU inference recipe for MiniMax H3 combining SGLang, step distillation, SageAttention2, Cache-DiT, and `torch.compile`. It reduces the reported latency for a five-second video from 348.8 seconds to 28.7 seconds, but the critical internal acceleration weights have not been released, and the public LoRA cannot directly reproduce that result.

2026-09-12

最新模型

T1 Replays MoE Routing to Stabilize Long-Horizon Reinforcement Learning, Raising Terminal-Bench 2.1 Success Rate to 64%

T1 records and replays MoE expert selections for every token, preventing numerical differences between rollout and training from sending them through different subnetworks. Its open weights support self-hosting, but benchmark results and competitor comparisons still come mainly from the team’s own evaluations.

2026-09-12

最新模型

X-AuT Progressively Prunes Qwen3-ASR’s Audio Tower, Cutting In-Vehicle Encoding Latency by 21.4%

XPeng uses behavioral probes, cross-scale distillation, and LoRA to shrink the Qwen3-ASR-0.6B audio encoder from 18 layers to 14. The weights and inference code are public, but the full distillation pipeline and more than 280,000 hours of training data have not been released.

2026-09-12

推論系統

vLLM-Omni 0.29 RC Brings Full-Duplex Voice into a Unified Runtime, but Barge-In Support Still Varies by Model

The new release serves MiniCPM-o 4.5, PersonaPlex, and Nemotron VoiceChat through a shared WebSocket runtime, while adding diffusion batching, parallelism, and component offloading. It remains a pre-release, and some endpoints silently fall back to turn-based voice, so a successful connection alone does not confirm full-duplex capability.

2026-09-12

AI research

Looped Flows Trains Iterative Reasoning Through Local Denoising, Reaching 58.8% Accuracy on ARC-AGI-1

The new method arranges each recurrent-network update into a continuous flow from noise toward an answer, allowing the model to learn state across steps without fully backpropagating through every iteration. At inference time, more integration steps can be used for additional compute, while different initial noise samples can produce multiple candidate solutions.

2026-09-12

最新模型

GPT‑Live‑1 API Decouples Full-Duplex Voice from Backend Reasoning, Pricing the Voice Layer at $0.05 per Minute

OpenAI is giving developers access to GPT‑Live‑1, which can listen, speak, and handle interruptions simultaneously, while allowing the voice front end to delegate complex reasoning and tool operations to other models. The new architecture reduces fragile handoffs between STT, LLM, and TTS, but backend reasoning, tools, and call infrastructure are billed separately.

2026-09-12

AI 安全

LiteLLM Testing Finds 9.6% of Public Gateways Accept Default Keys; MCP Flaw Can Turn Any Token Into a Valid Session

Wiz scanned 3,074 public LiteLLM instances and found that 294 accepted `sk-1234` or required no authentication at all, including 191 with no authentication configured. Newly disclosed real-world measurements show how an authentication bypass in older MCP implementations, Python-executing guardrails, and access to cloud metadata can be chained into a cross-layer attack path.

2026-09-11

最新模型

Nemotron Wins IMO Gold With a Three-Model Generate–Verify–Revise Pipeline, but the Full Competition Run Consumed 2.31B Tokens

NVIDIA has released the weights for two Nemotron 3 Ultra math specialists, their training data, and a resumable natural-language proof pipeline. The system’s official score was 30 out of 42. The results also expose a shared blind spot among model-based verifiers: internal evaluations overestimated performance on two problems, while high precision came at the cost of extensive false rejections and substantial compute.

2026-09-11

GitHub Repo

Edge0 Streams MoE Experts from SSD, Requiring Just 2.9 GiB of Active Memory for a 35B Model at Short Context Lengths

The new open-source framework uses predictive routing to preload experts for the next step and Recover-LoRA to compensate for int4 quantization loss, so the entire 23GB checkpoint does not need to remain resident in memory. Official M4 Pro tests measured 14.9–17.7 tokens per second, but the framework currently supports only Apple Silicon, and its memory and quality results have yet to be independently reproduced.

2026-09-11

推論系統

NVIDIA Dynamo Splits Multimodal Inference into Three Stages, Cutting Time to First Token by Up to 42.2% Under Mixed Traffic

Dynamo can now assign visual encoding, LLM prefill, and decoding to separate workers and different GPU tiers, preventing image requests from blocking text-only traffic. Although NVIDIA’s tests show improvements of up to 5× in time to first token and 7× in end-to-end latency, long outputs or large dense models may offset the benefits of disaggregation.

2026-09-11

推論系統

Delaying the Submission of Ready Turns in Agentic Workflow Scheduling Cuts P95 Completion Time by Up to 71.4% Under Congestion

New research separates a turn becoming ready from its submission to the inference engine, using CVaR tail risk and a dynamic workload budget to determine submission order. SWE-bench trace replays achieved up to a 3.5× P95 speedup, but the results cover only fixed agent trajectories, a single randomized arrival seed, and an A100/vLLM environment.

2026-09-11