Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
最新模型
Kolibri-1 is released with open weights under Apache 2.0. It activates about 3.46B parameters per token and targets German-English bilingual use, document processing, and tool calling. Its million-token window extends beyond the model’s native 256k training length; the company still recommends shorter contexts for complex tasks.
GitHub Repo
A reproduction test with a large vocabulary suggests that the vLLM V1 CPU sampler may deviate from the model’s original probability distribution. The issue involves a fixed Gumbel random table; the impact on real models and an official fix remain to be confirmed.
最新模型
Meituan’s new preview model retains a one-million-token context window and adds image understanding and coding-agent integrations. Official documentation lists inference controls, but the visual input format and performance on real tasks remain to be verified.
GitHub Repo
In a two-node H100 test, adding a KV connector that moves no data increased per-token latency by about 87% under a specific workload. The investigation points to response aggregation and thread waiting; the cause and any fix still await upstream confirmation.
GitHub Repo
A reproduction on MI355X shows that compilation may turn a probability that should be zero into one for a particular operation. The reporter provided a temporary workaround, but the scope of impact and an official fix remain unconfirmed.
GitHub Repo
The release preserves cloud OAuth state and pairs that change with SDK fixes for token refresh endpoints and state writeback. The full effect still depends on backend deployment, and existing connections need validation across conversations.
AI 研究
The collaboration released high-confidence complex predictions for 2,812 viral proteomes, along with a GPU computing pipeline. Deployment still requires large GPUs and cluster infrastructure, and predicted structures need experimental validation.
GitHub Repo
The system uses measured GPU kernel times to predict serving bottlenecks, then has an agent implement candidate optimizations in an inference framework. The team has released the code and test cases, but gains depend on the model and workload, and some patches are still awaiting upstream review.
AI 研究
The method has an agent read the full transcript, then re-transcribe suspicious segments to correct inconsistent proper nouns across segments. The official implementation is public, but the largest gains are limited to a specific model configuration; deployment still requires weighing the rate of incorrect edits against inference costs.
推論系統
The roughly 280-million-parameter draft model adds speculative decoding to LFM2.5-VL-3B and is available with public weights and framework integrations. Official tests cover specific hardware and settings; vision encoding, prefill, and concurrent workloads can still limit overall gains.
GitHub Repo
The new release uses a background thread to preload the next batch of images, reducing the time the GPU spends waiting for data. Published benchmarks cover specific YOLO26n configurations, while the release summary and merge history still differ on the feature’s scope.
AI 研究
The architecture uses two levels of KV sharing to end prefill early for long inputs. The fivefold difference comes from computational analysis; quality evaluations extend to 256k, while real-world serving latency remains unverified.