Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
推論系統
The Rust/CUDA inference engine Atlas has submitted MLPerf Inference 6.1 results using GB10 and AMD gfx1151 systems, testing multi-turn tool calling on a single accelerator. The submissions are public, but throughput and latency results remain under embargo, so no conclusions about cross-hardware performance can yet be drawn.
代理工具
Common Lisp programming agent Autolith released versions 0.36 through 0.38 within two days, adding the standard Agent Skills format, Nous and Mistral backends, and cross-platform distribution packages. Its recoverable, replayable self-modification design improves inspectability, but generated code still runs with the user’s privileges and must not be treated as a secure sandbox.
推論系統
llama.cpp 0.2.0 officially introduces semantic versioning for slower-moving stable releases, while retaining the existing `b[number]` tags for development builds produced for nearly every commit. The new system reduces versioning risks for Debian, Homebrew, and downstream bindings, but “stable” does not mean long-term support or comprehensive validation across hardware platforms.
最新模型
An independent test tasked a quantized Qwen3.8-27B with statically analyzing an unnamed commercial application; after roughly 30 minutes, it produced a working license-bypass program. This was only a single case that cannot be fully reproduced, but it suggests that even when a model initially refuses, it may gradually cross its original safety boundaries over a long tool-use trajectory.
最新模型
Ox Alpha has sparked a wave of testing on OpenRouter and OpenCode with its free access, long context window, and multimodal agent interface. OpenRouter’s current page explicitly states that the anonymous provider retains prompts and completions, making early claims of “zero data retention” unsuitable as a basis for production use.
最新模型
OpenAI has cut GPT-5.6 Sol API input and output prices by 20% and 33%, respectively, with the promotional rates lasting at least through November 21. Once input exceeds 272K tokens, the higher rates still apply to the entire request, so long-context agent costs cannot be estimated from standard rates alone.
代理工具
MidTool places APIs, MCP, documentation, code, and synthetic workflows in a dedicated stage after pretraining and before SFT. The dataset and model checkpoints are now public, but the models still score zero on deep-search evaluations, showing that learning schemas is not the same as acquiring long-horizon research capabilities.
推論系統
Instead of using fixed offloading, FreeToken uses measured bandwidth to decide whether to transfer cache-missed experts to the GPU or execute them directly on the CPU. The paper also demonstrates a 753B model served by a single workstation GPU, but the full weights still require system memory; this should not be interpreted as “fitting all 753B parameters on one GPU.”
生成式視覺與 4D 重建
The system uses a fixed-length reference context and view regrouping during diffusion to reduce structural drift when generating multiview videos in batches. The weights and inference pipeline are now public, but the fully open-source 4D Gaussian Splatting reconstruction and low-memory mode are not yet complete.
最新模型
Marin has begun training a mixture-of-experts (MoE) model with 535B total parameters and approximately 23B active parameters per token. The project plans to use 11 GB200 NVL72 systems to process roughly 18T to 18.75T tokens. Configurations, scaling experiments, and real-time telemetry are public, but model quality, the long-context strategy, and the final token budget have yet to be finalized.
推論系統
The new release processes checkpoint staging and CUDA Graph capture concurrently during startup, delivering a 2.38× speedup over the standard default workflow in official H100 testing. It also rewrites parts of the tensor-parallel communication path and adds seven model families, but the Torch upgrade and cache-path consolidation introduce migration costs.
代理工具
Hume AI and Hugging Face found that when audio conflicts with the answers in public test sets, some high-scoring speech models still output content from the reference transcripts. The Open ASR Leaderboard has added a Benchmark Fitting tab, but these behaviors do not prove that the models directly encountered the test data.