Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
GitHub Repo
A Transformers community report on October 15 says the v5 DeBERTa tokenizer may omit SentencePiece normalization rules, causing some compatibility characters to be encoded as unknown tokens. Another participant independently reproduced the issue, but an official fix and downstream accuracy impact remain unconfirmed.
GitHub Repo
An October 15 report in the Ollama community says an AMD GPU may run out of memory during HIP graph initialization when running a partially CPU-offloaded Gemma 4 model. The reporter temporarily avoided crashes by disabling graph execution, but maintainers have not confirmed the root cause or issued an official fix.
GitHub Repo
Hugging Face released Transformers 5.20.0 on October 15, adding quantization during export and distributed checkpoint improvements. The project also warned of a performance regression in expert parallel inference; deployments using legacy cache interfaces will need code changes.
GitHub Repo
A report from the Open WebUI community on October 15 says streaming replies rewrite the accumulated full text to Redis token by token, increasing transfer costs for long-running agent tasks. The report includes measurements using a simulated model and reproduction steps, but an official fix and the impact on production deployments remain unconfirmed.
GitHub Repo
Proposed on October 14, the Transformers patch centralizes ZeRO-3 checkpoint reads and weight conversion for non-quantized models on rank 0, reducing host memory pressure. The author included a two-process regression test and short training results on a large model, but the proposal has not been merged.
GitHub Repo
NousResearch released Hermes Agent v0.22.2 on October 15, correcting a build that mistakenly included unreleased gateway code. The update also addresses false timeouts on long prompts with local models and gateway startup failures after reboot, though some sessions created by the previous release still cannot be deleted.
GitHub Repo
A Transformers discussion on October 13–14 reported that enabling timestamp generation for Whisper with a single-element EOS list may trigger a type error. The reporter added a reproduction using real audio, but the fix’s merge status and impact on released versions remain unclear.
GitHub Repo
A Transformers community report on October 14 says SAM 3 skips the geometry encoder for text-only prompts, making the input structure differ from Meta’s reference implementation. Participants reported independent reproduction and proposed a fix, but maintainers have not confirmed the expected behavior, and any impact on segmentation accuracy remains unmeasured.
GitHub Repo
A Transformers report on October 13 says a newer tokenizer loading path may ignore the model’s existing normalization settings, adding a trailing newline token to WeMM-Embedding-2B inputs. The report includes a cross-version minimal reproduction, but maintainers have not confirmed it and retrieval quality has not been measured.
最新模型
On October 13, Liquid AI unveiled LFM2.5-24B-A2B and base model weights, extending context to 131,072 tokens and switching to a fixed reasoning-before-answering format. The company published local decoding and agent benchmark results, but deployers should still validate end-to-end task latency, tool parsing, and post-quantization quality.
GitHub Repo
A Dify community report on October 13 found that an LLM node may treat JSON in a reasoning block as structured output, even when the final answer matches the configured format. The proposed fix includes streaming and non-streaming regression tests, but has not been merged and does not add full JSON Schema validation.
GitHub Repo
A Transformers technical discussion on October 12–13 proposed using native GQA for CPU inference with attention masks, avoiding K/V tensor copies. The authors measured 1.21× faster decoding, while a second machine showed only 1.06×; the proposal has not been merged.