All News

1492 technical news articles

vLLM Community Proposes MoE LoRA Backend Fix; Explicit Triton Selection May Still Fail to Start on Blackwell

Reports from the vLLM community on October 12–13 say MoE LoRA backend selection may ignore an explicit setting and trigger a startup error in fully sharded mode. The proposed fix adds configuration checks and unit tests, but has not been merged and lacks full GPU validation.

2026-10-13

GitHub Repo

vLLM Community Investigates NVFP4 Cache Layout Gap; Interleaved Storage May Overlap Data and Scale Values

vLLM community reports from October 12–13 indicate that NVFP4 KV cache reads and writes may be misaligned under certain token-major layouts. Another participant reproduced data overlap at the kernel level. Maintainers have yet to clarify the supported configurations, so the issue should not be assumed to affect all production deployments.

2026-10-13

GitHub Repo

Ollama Pre-release Switches to Native Decision Inference in llama.cpp; Older GGUF Models May Need to Be Re-downloaded

On October 12, Ollama merged an update to its decision-model inference path and included it in v0.40.4-rc1, delegating prompt assembly and scoring to llama.cpp. The update also changes scoring requirements and model metadata, so existing deployments should verify model-file compatibility.

2026-10-13

GitHub Repo

Transformers Community Reports Duplicate Work During ZeRO-3 Loading, with Each Process Potentially Reading and Converting the Full Checkpoint

A report in the October 11 Transformers development branch says that enabling ZeRO-3 may still leave each process loading the full checkpoint, adding resource pressure during startup. The report includes a two-device minimal reproduction and results from a partial patch, but there is no official fix or comprehensive memory measurement yet.

2026-10-11

GitHub Repo

Transformers Community Proposes T5 Attention Fix as Tensor Layout May Send SDPA to Slower Kernel

A Transformers community report on October 12 says T5 positional-bias tensor layout may prevent SDPA from using fused kernels, slowing embedding workloads. A proposed fix includes AMD and NVIDIA tests, but has not been merged, and gains still need validation on real workloads.

2026-10-13

GitHub Repo

Ollama Community Reports Discrepancy in Prompt Throughput Calculation; Cache Hits May Inflate Performance Figures

A report posted to the Ollama community on October 11 says the OpenAI-compatible endpoint may divide the total number of prompt tokens, including cached tokens, by the processing time for uncached tokens, inflating prompt throughput. The report includes a streaming response and reproduction steps, but the fix status and cross-version impact remain unconfirmed.

2026-09-23

GitHub Repo

Transformers Community Investigates Packed-Training Boundary Gap; Linear-Attention Fallback May Mix in Previous Documents

Technical discussions in Transformers on October 12–13 pointed out that, when accelerated kernels are unavailable, fallback implementations for linear attention and convolutions may ignore document boundaries and alter packed-sample computations. The community has proposed two fixes—isolating sequences or raising an error—but neither has been merged.

2026-10-13

GitHub Repo

Dify Community Reports Agent Streaming Delay; Enabling Fail Branch May Hold Text Until Task Ends

A Dify community report on October 12 says that after an Agent node’s Fail Branch is enabled, text chunks may be buffered and sent only when the node completes. The report includes event timestamps and code path analysis, but has not yet been confirmed by maintainers or addressed by an official fix.

2026-10-12

GitHub Repo

Ollama Pre-Release Reuses Verified Models, Cutting Warm Embedding Request Latency by More Than 40%

On October 12, Ollama merged a model lookup optimization and included it in v0.40.4-rc0, reducing repeated processing for subsequent requests to the same model. Maintainer tests showed warm-request latency reductions of 41% to 56% for two embedding models, though the results still need validation in real deployments.

2026-10-08

GitHub Repo

Hermes Agent Community Investigates Missing Task Notifications After Reset; Background Processes May Be Marked Read

A report from the Hermes Agent community on October 12 said a session reset may terminate background processes without telling users they were interrupted. Another participant’s partial execution test traced the notification gap to a cleanup flag, but the full Telegram flow and a production fix remain unconfirmed.

2026-10-13

GitHub Repo

Transformers Community Reports Mamba2 Grouping Gap in Normalization; Non-Fused Path May Differ from Training Core Computation

An October 12 report to the Transformers community says non-fused Mamba2 RMSNorm does not normalize by configured groups, potentially producing different outputs and gradients from the fused training path. The report includes CPU and CUDA comparisons and a local patch check, but no official fix or full-model quality evaluation.

2026-10-12

GitHub Repo

Ollama prerelease fixes MLX rollback state, reducing memory retention after tool calls

Ollama merged an MLX memory fix on October 12 and included it in v0.40.4-rc0. The change addresses large buffers retained after speculative decoding rollbacks. In a 12-tool-call comparison, maintainers observed 3.52 GiB less memory held in the final round, though results across models remain unverified.

2026-10-06