Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
AI 基礎設施
The open-source GigaToken 0.9.0 project reoptimizes pre-tokenization, BPE merging, and the data input path, claiming a 989× speedup over Hugging Face Tokenizers on a dual-socket, 144-core AMD EPYC server. Its top result depends on the native API, an 11.9 GB corpus batch, and highly parallel hardware, and cannot be directly extrapolated to short online requests.
最新模型
South Korea’s Upstage has released Solar Open 2 for long-horizon agents, combining a 250-billion-parameter MoE, hybrid attention, and a 1-million-token context window in a single model. The company says the quantized model can be deployed on two NVIDIA H200 GPUs, though current evidence for its performance and long-context stability comes primarily from the team’s own testing.
推論系統
BaseRT has added hand-written dense and MoE GEMM and FlashAttention kernels, offloading matrix operations during the prompt-processing stage to the neural accelerators inside the M5 GPU. The team reports substantial leads over llama.cpp and MLX on the M5 Pro, although the figures currently come solely from the developers’ own single-machine tests.
推論系統
SWE-Pruner Pro replaces a separate classifier with a lightweight prediction head that uses a coding agent’s internal representations to decide whether to retain each line of tool output. Tests with two open MoE models across four multi-turn benchmarks showed token savings, but the method requires inference-server modifications, and some quality gains have yet to be reproduced.
Inkling enters the open-weight market with a Mixture-of-Experts architecture featuring 975 billion total parameters and 41 billion active parameters. The model also offers adjustable reasoning effort, a one-million-token context window, and a path to fine-tuning on Tinker.
The GPT-5.6 family is now generally available, with Sol, Terra, and Luna covering maximum capability, balanced cost, and high-volume, low-cost workloads, respectively. All three models share a long-context architecture and tool interface, but their inference costs, safety behavior, and real-world quality still require separate evaluation.
最新模型
Muse Spark 1.1 is available in public preview for the first time through the Meta Model API, with an OpenAI-compatible interface. The model combines a one-million-token context window, search citations, structured outputs, and parallel tool calling into a foundation for agents.
推論系統
Diffusers can now load Nunchaku Lite INT4 and NVFP4 checkpoints directly, without requiring a separate model-specific inference engine. SVDQuant quantizes both weights and activations, aiming to genuinely shorten denoising time in addition to saving VRAM.
應用研究
UniVR seeks to jointly learn complex reasoning, physical dynamics, and long-term planning solely from visual demonstrations. ByteDance has also released the 34B Planning checkpoint, providing a testable baseline for research on visual agents that do not rely on textual chains of thought.
推論系統
Nemotron 3 Embed offers text embedding models in 8B BF16, 1B BF16, and 1B NVFP4 variants. The low-precision version treats memory efficiency and throughput as first-class design goals for retrieval models, but its quality still needs to be validated on target corpora.
GitHub Repo
Google has released the open-weight Gemma 4 12B model, replacing separate vision and audio encoders with linear projections. The new architecture simplifies fine-tuning and local inference, but the official performance figures still require independent verification.
Microsoft plans to deploy AMD Helios at scale on Azure and launch HDv2 and HXv2 instances powered by sixth-generation EPYC processors, along with MI455X inference instances. The infrastructure update co-designs GPUs, CPUs, DPUs, networking, and software, though its real-world price-performance remains to be demonstrated through public testing.