All Articles · SURL AI News

vLLM Community Reports Ignored Per-Layer LoRA Scaling, Potentially Degrading Fine-Tuning Results in Deployment

A report says vLLM 0.30.0 does not apply PEFT’s per-layer rank and alpha settings, which could silently change adapter behavior. The community has provided a reproduction and a weight-conversion comparison; an official fix and the scope of impact remain unconfirmed.

2026-10-03

GitHub Repo

Ai2 Open-Sources Olmo-core 3, Redesigning Expert Placement and Data Routing for MoE Training

The new version keeps experts resident on GPUs and uses data routing to reduce the cost of repeatedly aggregating weights. Initial tests reported about 2.7× throughput. A trillion-parameter configuration has passed systems testing, but it used random routing, so training quality and stability remain unverified.

2026-10-03

Cloudflare Open-Sources Clef Decision Models, Producing Option Probabilities in a Single Forward Pass

Clef and Clef-flash turn agent classification and routing into direct scoring of specified options, and release their weights under Apache 2.0. Official tests show latency advantages, but task accuracy and probability calibration still need to be validated against deployment data.

2026-10-03

GitHub Repo

Open WebUI Discloses Tool Export Vulnerability; Read-Only Shares May Leak Source Code

Non-admins with tool export permission could download the Python source code of tools shared with them as read-only. The issue was fixed in 0.11.4; operators should review group permissions and check whether credentials are embedded in code.

2026-09-28

GitHub Repo

Vercel Perplexity SDK 5.0.0 switches to the Agent API; search and streaming integrations must migrate

The new version moves language generation to the Agent API and changes model configuration, streaming events, and usage data formats. Existing PDF inputs and image and video results are no longer supported, while official sources still disagree about the Sonar deprecation timeline.

2026-09-28

GitHub Repo

PyTorch Precompilation Proposal Packages Kernels, Strengthens No-Compile Checks During Replay

The author reports that the full set of changes let 16 distributed processes replay a recommendation model without triggering compilation. The proposal is still a draft; validation across models and deployment benefits remain to be confirmed.

2026-09-28

GitHub Repo

vLLM Community Reports Multi-Node Decoding Latency; Even an Empty KV Connector May Reduce Performance

In a two-node H100 test, adding a KV connector that moves no data increased per-token latency by about 87% under a specific workload. The investigation points to response aggregation and thread waiting; the cause and any fix still await upstream confirmation.

2026-09-28

GitHub Repo

vLLM Community Reports Output Divergence After Restart; Core Autotuning May Affect Reproducibility

Tests with Qwen3-8B, a fixed seed, and zero temperature still produced different outputs when core configurations varied. A related PyTorch patch has not been merged, and the scope of the issue needs to be checked against the version, hardware, and compilation path.

2026-09-27

GitHub Repo

Dify Community Reports OpenAPI Parameter Inheritance Bug; Custom Tools May Send Incorrect Paths

Shared path-level parameters may be lost during import, and Swagger-formatted specifications may be rejected outright. The reported cause can be cross-checked against the source code and specification, but the fix’s release status and full impact remain unconfirmed.

2026-09-27

GitHub Repo

PyTorch Community Reports Numerical Anomaly in ROCm Compilation; Certain Probability Calculations May Be Distorted

A reproduction on MI355X shows that compilation may turn a probability that should be zero into one for a particular operation. The reporter provided a temporary workaround, but the scope of impact and an official fix remain unconfirmed.

2026-09-27

GitHub Repo

vLLM Community Test: Enabling Pinned Memory in WSL2 Cuts Single-GPU Decoding Latency by About 12%

A single-machine test suggests that changing a memory setting can reduce GPU wait time during decoding. The default remains disabled; benefits and capacity limits should be validated in each deployment environment.

2026-09-27

GitHub Repo

vLLM Community Traces H20 MoE Scheduling Inefficiency to Expert-Load Kernel Selection

Community tests suggest FlashInfer’s workload estimate may cause the H20 to select an inefficient compute kernel. After adjustment, a specific MoE operator ran about 3.21× faster. The patch has not been merged, and overall serving gains and some correctness tests remain unverified.

2026-09-27