Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
最新模型
The Shanghai AI Laboratory has released Atria Dawn Preview, along with FP8 weights and inference recipes, positioning it as a full-loop agent capable of searching, coding, and running experiments. The organization claims the best reported results on five of 16 benchmarks, though model scale, test-time compute, and its in-house evaluation setup still limit comparability.
代理工具
GitHub released 17 `gh-aw` versions in one week, enabling JSON logs to identify the server, tool, and timestamp for each MCP call. The latest patch also stops persistently storing `GITHUB_TOKEN` in the slash command router’s Git configuration, though the 0.89 series remains on a prerelease cadence.
推論系統
A new study introduces SQD, which assigns decoding workloads according to the memory and compute characteristics of quadratic and subquadratic attention. The paper reports up to a 56% gain in tokens per joule, though its Rubin/LPX figures still come from analytical models rather than a physical cluster.
代理工具
In a concentrated public promotion push on September 13–14, Otis recommends and downloads GGUF models based on the user’s hardware, then launches a local agent using its bundled llama.cpp. Offline mode requires no account, but search, PAIR clusters, and Fireworks inference operate across distinct data and trust boundaries.
端側 AI
Apple released the final versions of iOS 27 and its other operating systems on September 14, bringing the new AFM 3-powered Siri to supported devices. The larger on-device model uses prompt-level sparse routing with NAND-based weight storage, a technically distinct approach from typical MoE models that select experts for each token.
AI 安全
K-Bench expands evaluation beyond a model’s final response to six agent channels, including reasoning, tool calls, and retrieval results. Experiments show that even when existing benchmarks report no leakage, deployed agents may still expose target data in 22% to 86% of queries.
代理工具
Reverse-engineering experiments show that macOS 27’s `Inference Providing` path can pass Siri’s planning prompts, tool definitions, and execution results to an external model. The demo requires disabling system protections and using a private Apple entitlement, so it should not be interpreted as meaning users can now officially replace Siri’s model.
推論系統
Architecture support merged on September 14 allows Maple-Preview to be loaded and converted without relying on DeepGrove’s llama.cpp fork. The model activates only about 1 billion parameters per token, but GPU backends, quality evaluations, and official speed claims still require separate verification.
代理工具
New experiments show that relative terms such as `previous` in “draft–validate–revise” pipelines may change referents as model roles shift. Increasing reasoning effort can dramatically improve some models, but model selection matters more than simply adding test-time compute.
GitHub Repo
OpenArch rewrites models including Llama, DeepSeek, Gemma, Qwen, and Kimi as architecture-specific, pure PyTorch examples, making it easier to compare attention mechanisms, normalization methods, and MoE routing. The project quickly gained attention on Hacker News on September 14, but most models remain unfinished, and the existing code explicitly does not target performance or compatibility.
推論系統
The new open-source benchmark runs small models on Jetson devices, Macs, Raspberry Pis, and Android devices, measuring decoding speed, tokens per joule, latency, and temperature instead of focusing solely on data-center GPU throughput. Its creator says roughly 1,000 Jetson configurations have been tested, but the public leaderboard currently loads no data, leaving users to verify results through repository reports and raw artifacts.
GitHub Repo
The September 13–14 llama.cpp prerelease builds add a CUDA F32 path for systems where BF16 is unavailable, preventing outright failures on some older GPUs. Another fix reports an error when Nemotron-H NextN/MTP metadata is incomplete instead of triggering SIGFPE while loading the model.