Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
GitHub Repo
A reproduction on the October 7 development build suggests that appending multiple tokens after reusing a prefix may leave more than the configured threshold in the full-precision cache. The report includes a test that requires no weight downloads, but maintainers have not confirmed the issue, and GPU memory and output-quality effects have not been verified.
最新模型
On October 7, Liquid AI released d1-3B and the experimental d1-omni-600M, which return structured decisions without generating answers token by token. Local and Jetson deployment examples are available, but the low-latency figures depend on warm-up conditions, and evaluation results are still primarily based on the company’s own testing.
最新模型
On October 6, Google released EmbeddingGemma 2, which maps text, code, images, video, and audio into a shared vector space and makes its weights available under Apache 2.0. The model can load encoders as needed, but quantized memory figures and full-precision evaluation results should be interpreted separately.
GitHub Repo
An October 7 Ollama community test found higher latency for long-prompt requests with three decision models using MLX quantized formats on M5 Pro than with BF16. The report includes measurement methods and reproduction instructions, but maintainers have yet to confirm whether dequantization is a bottleneck.
GitHub Repo
A community report dated October 6 says image requests to Qwen3-VL may repeatedly crash the inference process when another large model is already using both GPUs. The report includes a core-dump stack trace, but maintainers have yet to confirm whether memory pressure caused a null-pointer error.
GitHub Repo
A community report on October 6 says that after Hermes Agent connects to a custom model proxy, cached context length and manual overrides may prevent the proxy from starting. A fix for the ineffective override has been proposed, but remains a draft and does not address cache expiration.
GitHub Repo
A PyTorch community reproduction on October 5 found that Inductor may drop an addition’s alpha coefficient under a specific combination of clipping and scaling. A draft fix was proposed on October 6 but has not been merged; the impact on full training runs remains to be verified.
GitHub Repo
A community report on October 5 says Ollama may reuse output indexes and omit a message completion event when it emits text before a tool call. The proposed fix includes regression tests but has not been merged or validated with actual model inference.
GitHub Repo
A community reproduction on Ollama 0.35.1 showed that the generate endpoint may accept valid JSON followed by non-JSON text, return success, and run the model. A fix for full-request validation was proposed on October 3 but has not been merged.
GitHub Repo
A community report involving Ollama 0.35.1 suggests that per-layer quantization settings for MLX models may not be applied correctly, causing tensor shape mismatches during loading. The model configuration confirms mixed-precision settings, but the root cause and the scope of impact on other models remain unconfirmed upstream.
GitHub Repo
A reproduction on the vLLM development branch shows that, even with batch invariance enabled, model metadata caching and module import order may still change inference results. The community has proposed a fix, but it has not yet been merged; the impact on released versions and other hardware remains to be confirmed.
GitHub Repo
A reproduction on Transformers 5.8.1 suggests that certain DeepSeek models may load the wrong tokenization pipeline, causing Chinese input to disappear and English spaces to be lost. Current development-branch documentation describes a fallback for configuration mismatches, but whether this case is fixed across versions remains unconfirmed.