推論系統

推論 runtime、量化、 serving、KV cache、成本與效能。

LLaDA-Image Unifies Image Generation and Editing in a Single 6B Diffusion Model, With Turbo Cutting Sampling to 4 Steps

Ant Group’s inclusionAI has released LLaDA-Image weights and Diffusers inference code. A single model handles text-to-image generation, reference-image editing, and Chinese and English text rendering. The distilled version uses Twin-DMD to reduce the recommended sampling process from 50 steps to 4, but training code and independent performance validation are still unavailable.

2026-09-04

推論系統

Magnitude Selects Local Models Based on Memory Bandwidth and Capacity, Automatically Configuring Quantization and Speculative Decoding for Coding Agents

The new open-source inference server Magnitude benchmarks hardware before recommending a complete combination of model, quantization, context length, and estimated speed. It automates the memory and concurrency settings most often misconfigured for local agents, though performance and model quality currently still depend largely on the project’s catalog and testing.

2026-09-04

AI 平台與基礎設施

NVIDIA Acquires Hugging Face for $12.93 Billion, Pledges to Maintain Multi-Cloud and Multi-Accelerator Support

NVIDIA has agreed to acquire Hugging Face, gaining control of a key distribution hub for models, datasets, applications, and inference services. The company has pledged that the platform will not be tied to NVIDIA hardware, but its governance, ranking practices, and cross-hardware optimization after the deal will require continued scrutiny.

2026-09-04

NeoMME Encodes Text and Images with a Single Bidirectional Transformer, Compressing Visual Document Indexes to 6 kB per Page

Hcompany has open-sourced the 260M- and 800M-parameter NeoMME models, which process text tokens and raw image patches with the same encoder instead of attaching a vision tower or causal decoder. Its retrieval variants produce both dense and late-interaction representations, although the current efficiency and accuracy figures come primarily from evaluations conducted by the authors.

2026-09-03

推論系統

NVIDIA PAIR Routes Requests to Idle Computers Through a Compatible Proxy, Distributing Local Multi-Agent Inference Across Nodes

PAIR adds a compatible proxy layer between Ollama, LM Studio, and agents, routing independent requests to computers on the local network according to model availability and GPU load. It increases workload concurrency; it does not pool VRAM or split a single inference across multiple GPUs.

2026-09-03

推論系統

HeadWiseKV Configures Long-Context Caching per Attention Head, Extending Qwen3.6-27B’s Usable Length to 161K

HeadWiseKV stops retaining the same amount of history for every attention head. Instead, it uses offline calibration to assign multilevel windows to individual heads and actually reduces physical KV residency. In fixed-model tests, the paper extends the maximum successfully validated context from 114K to 161K, although the higher figures reported by the public repository rely on specific quantization settings and a single RTX 4090D.

2026-09-03

硬體

mzCache Restores Mobile LLM Memory in Parallel Across CPU and GPU, Cutting Time to First Token by 2.1× to 5.5×

The MobiCom 2026 study divides model weights and the KV cache into shared buffers, allowing the GPU to begin prefill while the CPU restores evicted data. The work targets recovery latency after multitasking on smartphones, rather than improving the model’s generation speed itself.

2026-09-03

推論系統

Hugging Face Open-Sources 207 WebGPU Kernels, Turning Browser AI Operators Into Versioned Packages

`@huggingface/kernels` can dynamically load GPU operators from the Hub, complete with interfaces, tests, and WGSL implementations. In operator-level tests on an Apple M4, Hugging Face reported a 2.57× geometric-mean speedup, but the results exclude model loading, compilation, and data transfer costs.

2026-09-03

TranslatePsy Open-Sources African-Language Translation Models, With Quantized 0.8B to 4B Versions for Offline Use

Tether AI Research has released AfriSLM, fine-tuned from Qwen3.5, alongside the 17M-to-43M-parameter AfriNano models designed for browsers and smartphones. The models cover up to 19 sub-Saharan African languages, but claims that they outperform larger baselines are based primarily on automated metrics and have yet to be validated by native-language experts.

2026-09-03

推論系統

Unsloth 0.1.806 Enables MTP Speculative Decoding by Default, Making Local Inference with Qwen3.8 and GLM-5.3 Up to 2× Faster

Unsloth now enables multi-token prediction by default for Qwen3.8-Flash-Next and GLM-5.3-Flash, while improving its MLX, multi-GPU, and ROCm paths. The official claim of up to 2× higher speed is not accompanied by a complete benchmark matrix, while Qwen’s technical documentation reports gains of 1.3× to 1.7× at low concurrency.

2026-09-03

代理工具

Android Studio Quail 4 Stable Ships with 23 Agent Skills, Gemma 4 Enables Offline Cross-File Refactoring

Google is bringing automatic selection of Android-specific skills and on-device Gemma 4 inference to Android Studio’s stable channel. Code can remain on the developer’s machine, but local execution does not mean the agent has robust permission isolation or verifiable quality.

2026-09-03

代理工具

Coder Agent Relay Separates Cloud Reasoning from Self-Hosted Execution, Bringing Cursor Tool Operations into Enterprise Workspaces

Coder lets Cursor’s cloud agent retain the external control loop while moving command execution, file access, and tool operations into enterprise-managed workspaces. This layered architecture can apply existing RBAC, firewall, and audit controls, but reasoning and model traffic still pass through the vendor, so it should not be considered a fully private deployment.

2026-09-03