Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
推論系統
PELM goes beyond adjusting processor frequency: it dynamically selects speculative decoding configurations and verification depth, jointly controlling hardware and model workload under thermal constraints. The paper reports speedups of up to 23.1% and energy savings of up to 52.4%, but these peak results cannot be directly extrapolated to smartphones or other models.
推論系統
SageMaker real-time inference endpoints now support `PREFIX_AWARE`, which aims to route requests with the same system prompt, documents, or conversation history to the same instance. AWS’s official seven-node tests show substantial gains for long-prefix workloads, although serialization differences, popular prefixes, and cache capacity can still affect real-world performance.
地球觀測與應用研究
Google and NASA JPL enable a vision Transformer to directly combine EMIT’s spectral and spatial context, simultaneously estimating concentrations, plume boundaries, and emission-source locations. The publicly released model recovers about 80% of expert-labeled cases, but its higher sensitivity also introduces risks of surface confusion and false positives.
生成式音訊
YuE2 combines symbolic melody and chord planning, semantic tokens, and flow-matching audio synthesis in a single generation pipeline. Its weights can run on a 24GB GPU, but its noncommercial license and best-of-8 evaluation mean comparisons with commercial systems should still be interpreted cautiously.
代理工具
Φ-Bench expands evaluation beyond individual GPU kernels to multi-file implementation and end-to-end system optimization. All eight frontier models occasionally find effective modifications, but larger reasoning budgets do not consistently translate into better engineering outcomes.
最新模型
Built on the 2.8T-parameter Kimi K3 base model, SWE-2 improves coding-agent efficiency through cost penalties, low-precision rollouts, and a draft model updated in sync with training. Official benchmarks show performance approaching that of frontier closed-source models, but the results were largely produced by Cognition itself, and neither a standalone API nor model weights are currently available.
最新模型
North Small Translate activates 25B parameters per token out of 218B total and supports more than 50 languages and regional variants, including Traditional Chinese. The model can be tested through the Chat V2 API or self-hosted using its FP8 weights, but CC BY-NC 4.0 prohibits direct commercial use, and Cohere has not published comparable translation benchmark results.
最新模型
Hugging Face fixed unnecessary FP32 casting in `chunked_nll` and demonstrated a single training step on a one-million-token sequence using eight H100 GPUs. The new release also completely removes the legacy PPO interfaces and raises the minimum versions of several dependencies, requiring teams to recheck training code and numerical behavior before upgrading.
AI for Science
Google DeepMind has precomputed AlphaGenome predictions for every possible single-nucleotide substitution across the human genome, creating a 1 PB atlas that uses AVI scores to rank coding and noncoding variants on a unified scale. It sharply reduces the cost of large-scale screening, but the data remains model-generated and cannot replace functional experiments or clinical diagnosis.
最新模型
Inception has released Mercury 2.5 with a 260,000-token context window, adjustable reasoning, and parallel tool calling, continuing to replace purely autoregressive decoding with iterative refinement of multiple tokens. The company claims 1,107 tokens per second, but Vercel AI Gateway reports a real-world median of about 616 tokens per second, so deployers should still retest it with their own prompts and concurrency conditions.
GitHub Repo
OpenBMB has released a 2B dense model with a 131K-token context window and tool-calling capabilities, along with staged checkpoints, training data, and multiple quantization formats. Third-party evaluations affirm its competitiveness within its class, but community testing suggests the official sampling settings may cause GGUF quantizations to fall into repetitive generation loops.
代理工具
The new release lets supported models continue using the KV cache when the system prompt is dynamically updated, reducing recomputation costs in long-running sessions. The tradeoff is a set of backward-incompatible changes across Session V3, Agent, and Inbox APIs, requiring plugins and custom log tooling to be revalidated.