SURL AI News
繁中English日本語

Archive

推論系統

推論 runtime、量化、 serving、KV cache、成本與效能。

代理工具

JarvisHub Turns the Canvas Into Shared State for Multimodal Agents, but Its First Release Still Lacks Quantitative Evaluation

The open-source JarvisHub uses typed nodes and edges to preserve assets, versions, dependencies, and agent trajectories, allowing long-horizon creative work to live outside linear conversations. The system already provides an executable three-layer architecture and local deployment tooling, but the paper currently presents only qualitative case studies and has yet to demonstrate improvements in reliability or output quality.
推論系統

Sol-Attn Dynamically Sparsifies Attention Within Online Softmax, More Than Doubling End-to-End Video Generation Speed

NVIDIA researchers have unveiled Sol-Attn, a training-free technique that selects key blocks during attention computation and approximates the contributions of skipped blocks. In NVIDIA's tests across several video models, it delivered roughly 1.9× to 2.34× end-to-end speedups, although its quality and generalization across hardware still require independent verification.
推論系統

NVIDIA Dynamo Provides Multi-Node Deployment Recipes for Kimi K3; Million-Token Inference Requires at Least 16 GB200/GB300 GPUs

An experimental Dynamo release adds multimodal processing, reasoning, and tool-call parsing for Kimi K3, along with Kubernetes recipes for aggregated inference and disaggregated prefill/decode. The recipes quantify the deployment threshold for the 2.8T-parameter MoE model in terms of rack topology, but they have not undergone formal QA, and the GB200 configuration still has a known startup error.
代理工具

Grok 4.5 Comes to GitHub Copilot With a 500K Context Window and Cross-IDE Reasoning Controls

GitHub has begun rolling out Grok 4.5 across Copilot’s IDE integrations, CLI, cloud agent, and desktop app, offering a context window of up to 500,000 tokens, text and image input, and three reasoning effort levels. Enterprise administrators must explicitly enable the model policy, while GitHub has so far disclosed only observations from internal terminal-based tasks, with no comparable Copilot benchmark results.
代理工具

GitHub Copilot Opens Agent Telemetry and Token Limits in JetBrains, While BYOK Endpoints Gain Centralized Model Controls

GitHub has updated its JetBrains plugin to let agent workflows export OpenTelemetry data and to support input and output token limits for bring-your-own-key configurations and custom endpoints. Claude agent flows can also connect directly to MCP servers and custom agents, but deployment teams still need to validate tool-call content, cost attribution, and data redaction.
硬體

ESP32-S3 Runs a 28.9-Million-Parameter Model from 16MB Flash Using Hierarchical Embeddings, Generating About 9.5 Tokens per Second

The open-source esp32-ai project moves most parameters into memory-mapped Flash while keeping only a small Transformer core in fast memory, enabling a roughly $8 microcontroller to generate text entirely offline. The result shows that hierarchical models can be embedded in firmware, but TinyStories-level output should not be equated with general language understanding or device-control capabilities.
推論系統

HPD-Parsing Splits Document Generation into Parallel Branches, With the 1B Model Reaching Peak Throughput of 4,752 Tokens per Second

PaddlePaddle has released the 1B-parameter HPD-Parsing model, which uses a global layout branch to coordinate multiple block decoders and bypass the sequential bottleneck of generating an entire page token by token. Official tests show throughput 2.62 times that of the fastest existing comparison model, but the system currently depends on a custom vLLM package, and its peak figure cannot be directly equated with end-to-end PDF processing speed.