推論系統

推論 runtime、量化、 serving、KV cache、成本與效能。

VLM Run Gateway Unifies OCR, Video, and Vision Model APIs, but Model Names Still Do Not Guarantee Equivalent Inference

VLM Run has launched an OpenAI Chat Completions-compatible gateway that lets OCR, image captioning, video understanding, and specialized ViTs share endpoints and an MCP interface. It aims to expose differences in quantization and serving configurations, but current claims about performance, visual accuracy, and provider support rates are based primarily on the publisher’s own testing.

2026-09-08

最新模型

Tencent’s EVIE Cuts Visual Document Retrieval Index Costs with Truncatable Multi-Vectors and Token Clustering

Tencent has released EVIE-8B and EVIE-4.5B, which create per-token representations directly from page images instead of performing OCR before retrieval. The 4.5B model supports runtime-adjustable vector dimensions and can compress each page to 32 vectors, although its leading results currently come mainly from the team’s own evaluations.

2026-09-08

GitHub Repo

InvokeAI 6.14.1 Patches the Trust Boundary for External Image Downloads and Corrects VAE and FP8 Memory Reservations

InvokeAI’s patch release tightens trust checks for Alibaba Cloud image downloads and injected sessions while correcting memory estimates for FP8, FLUX.2, and VAE workloads. The new version also supports Krea-2 LoRAs with DoRA magnitude, but the project has not provided a vulnerability identifier or quantifiable VRAM comparisons.

2026-09-07

AI 基礎設施

LiteLLM 1.100.0 Brings Shared Budgets, MCP Token Validation, and Vertex Interactions into a Single Gateway

The new release makes shared budgets for model access groups persistent across billing cycles and adds RS256 signatures and RFC 7662 introspection for MCP sessions. It also expands routing for Vertex AI, Bing grounding, and speech models, but removes an existing token calculation function, so direct imports and database migrations must be checked before upgrading.

2026-09-07

AI 研究

RISE Synthesizes a Per-Token Teacher by Extrapolating the RLVR Update Direction, Eliminating External LLM Distillation

The Salesforce AI Research team extrapolates the update displacement between the current model and a lagging anchor to synthesize a “future-policy” teacher that evolves throughout training. The method outperforms vanilla GRPO on mathematics and agent benchmarks, but requires 1.3× to 1.6× more wall-clock time and relies on the assumption that the local training trajectory is approximately linear.

2026-09-07

最新模型

Molecular Benchmark Audit of 22 Frontier Models: Higher Reasoning Effort Makes Published Values Easier to Retrieve

Researchers found that some models may not be making predictions in molecular property regression tests, but instead reproducing answers from their training data digit for digit. Retrieval signals increased by 89% at higher reasoning effort, suggesting that reasoning settings themselves can affect the extent of benchmark contamination.

2026-09-07

推論系統

Speculative Macro Commit Pre-Executes Agent Toolchains, Cutting AppWorld Wall-Clock Time by 44.9%

New research enables a small draft model to pre-run multi-step tool operations in isolated environment snapshots, then commit the entire sequence once a large authoritative model confirms the first step. The method reduces latency on two agent benchmarks, but requires three GPUs and lowers AppWorld task completion from 41.67% to 40.48%.

2026-09-06

硬體

Extropic Z1T Rewrites the Transformer for a Probabilistic Chip, but Its 100× Energy-Savings Claim Still Excludes Key Costs

Z1T maps part of the decoding workload onto Extropic’s Z1 probabilistic-bit chip using fixed four-way sparse projections, Dynamic Tanh, and convolutional attention. The code and one set of weights are public, but the energy-efficiency and throughput figures come primarily from hardware models rather than complete end-to-end measurements.

2026-09-06

推論系統

SGLang 0.5.19 Adds Beam Search and Restructures Inference Paths Across NVIDIA and AMD

The latest SGLang release can return multiple candidate sequences from the same service using `beam_width` and makes the unified radix tree the default cache for all models. The update also expands MoE, long-context, and ROCm kernels, but introduces dependency and configuration behavior changes that require revalidation before upgrading.

2026-09-06

最新模型

Auxiliary Views Experiment: Recasting the Same Knowledge as Textbooks, Q&A, and Articles Helps Models Learn Better Than Repeating the Original Text

Under a fixed knowledge-token budget, University of Pennsylvania researchers replaced some repetitions of original texts with multiple conceptual representations, improving factual recall and reasoning. The gains appeared mainly in models with 7B parameters or more, but the experiments involved only short-run continual pre-training and cannot be directly extrapolated to the full pre-training of frontier models.

2026-09-06

推論系統

Ollama 0.34 RC Connects Local Models to ChatGPT Desktop, Adds Compatibility for Tool Search and Response Compaction

Ollama’s macOS app can now connect local open models directly to the latest ChatGPT Desktop. It remains a release candidate, and the community has reported an interface issue when switching back to cloud models, making it unsuitable for immediate large-scale deployment.

2026-09-06

最新模型

Artificial Analysis Index 4.2 Doubles Private-Test Weighting to 40% and Removes the Saturated GPQA Diamond

The updated index adds benchmarks for agentic knowledge work and long-form PDF reasoning, raising the weight of non-public questions from 20% to 40%. This reduces opportunities to optimize for public benchmark sets, but also makes it harder for external researchers to fully reproduce the rankings.

2026-09-06