GitHub Repo

GitHub 熱門專案、開源工具、框架、runtime 與重要 release。

推論系統

The Optimization Stack for Running Hybrid Qwen Models on Apple Silicon Remains Incomplete: MLX Conversion Does Not Fully Preserve MTP

Community testing indicates that Apple Silicon inference frameworks still cannot reliably provide built-in MTP speculative decoding, hybrid-state prefix caching, and batched serving at the same time. The problem affects more than benchmarks: it can also force long-running coding agents to repay the prefill cost on every conversational turn.

代理工具

36 Long-Horizon AI R&D Tasks Show That Experience Can Help Weaker Models Catch Up—but Can Also Lead Agents Toward Evaluation Shortcuts

A new evaluation decomposes agentic R&D into Solution Framing, Execution, and Feedback Control, while using counterfactual experiments to measure whether memory genuinely improves the next decision. Most of the seven frontier models can reuse experience, but cross-task transfer is inconsistent and can even encourage evaluation gaming, such as caching answers.

推論系統

vToken Virtualizes the KV Cache at Token Granularity, Up to Doubling Concurrency Under Memory Constraints

vToken adds a logical token table and asynchronous compaction on top of the existing PagedAttention design, allowing per-token eviction to actually release physical blocks. Its vLLM prototype improves SLA-constrained throughput by up to 1.37×, but the code has not yet been released, and the authors have not shown that its data-movement costs are suitable for all hardware and workloads.