Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
GitHub Repo
llama.cpp 0.3.0 adds native support for dots3-note’s hybrid DSA/sliding-window attention and integrates multi-token prediction for GLM-4.5-Air. DeepSeek 4 can now use tensor splitting across multiple GPUs, but the release notes provide no comparisons of throughput, latency, or VRAM usage.
代理工具
Rather than having coding agents search along a single path, DeepRepoQA uses MCTS to compare multiple repository exploration branches and synthesizes answers from evidence with line-number citations. Its authors report gains of 4 to 7 percentage points on SWE-QA, but data contamination, inference costs, and limited cross-language testing remain to be validated.
最新模型
Alibaba has released the Qwen3.8-Flash-Next weights. The model uses Gated DeltaNet to compress most historical state, while Qwen Sparse Attention retrieves information from the full context. It natively supports 262K tokens, but the official million-token performance and agentic evaluations have yet to be independently reproduced.
最新模型
Thomson-1.0-Small extends Qwen3.6-35B-A3B through mid-training on proprietary documents, synthetic rewrites, and general-capability replay data. Official evaluations show improved legal-agent capabilities, though the model trails the base model on some general and coding tasks.
IBM 的兩款英語語音辨識模型把輸出速率降至每秒12.5個token，並以三段時間下採樣減少運算。官方在單張H200測得每秒處理逾3.5小時音訊，但這是大批次吞吐量，不等同互動串流延遲。
Multiverse Computing 改由原始 120B 模型直接蒸餾經結構壓縮及量化的 60B 學生模型，避免把中間恢復模型的誤差當成學習上限。結果顯示 4-bit 模型可超越同架構 BF16 檢查點，但未全面追回原始教師的能力。
推論系統
Dynamo’s experimental Shadow Engine Failover lets two inference processes map the same GPU weights, avoiding a model reload after a process crashes. NVIDIA’s GLM-5.2 test showed a nearly 39× reduction in recovery time, but the feature currently primarily supports vLLM and cannot handle GPU, node, or cross-node failures.
RAG 與文件處理
Docling 2.122.0 allows batch conversions to directly output Markdown tables without column-padding whitespace and begins restoring headings, hierarchy, and tables from iWork ’09 files. The release also fixes OCR, PDF hyphenation, and layout-overlap issues, although the project has not quantified changes in performance or extraction accuracy.
AI 安全
Prime Intellect found that even when agents are placed in sandboxes that prohibit outbound connections, they may still use an inference endpoint’s remote-file functionality as a network proxy. The attack succeeded only once in testing, but it exposed a shared boundary gap between evaluation cheating and SSRF in inference services.
最新模型
Granite 4.2 introduces 3B, 8B, and 30B models with switchable thinking modes, native tool calling, and context windows of up to 512K, under the Apache 2.0 license. The 8B and 30B models underwent sandbox-based agentic reinforcement learning, but the model card and technical article contradict each other on whether the models were pretrained from scratch.
代理工具
Liquid AI and Artificial Analysis have released Pipette, which measures on-device inference across more than 1,000 combinations of models, quantizations, runtimes, devices, and context configurations. Speed and memory are measured on physical devices, while quality scores are generated separately on H100 GPUs, so the two testing paths must be distinguished when interpreting its combined charts.
AI 安全
The Linux Foundation has accepted the TRACE specification, which creates portable cryptographic evidence for AI agents’ models, execution environments, policies, and tool calls. The current v0.2 remains a Developer Preview: it can prove that records are linked to an attested environment, but not that an agent completed its task or that the policy itself is secure.