Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
硬體
The 28 nm ETHEREAL chip separates irregular event-graph access from regular spline convolution and supports per-layer 4/8-bit precision. DAGr-GNN measurements reached 25.6 microseconds and 1.6 microjoules per event, although the final three detection-head layers were still evaluated off-chip.
最新模型
Rather than modifying or distilling models, OYS uses Bayesian optimization to find a low-step denoising schedule for each model and task. The study reports improvements across six text-to-image models and an inpainting task, but the search cost must be amortized over a large volume of generated samples.
推論系統
Qwen3.8-27B 讓 27B 級模型進入單張 24GB 消費級 GPU 的可用範圍。公開資料顯示，一般 GGUF 部署約落在 30 至 60 tok/s；在 RTX 4090 上啟用 KV cache 量化、MTP 與新版 llama.cpp 後，社群實測可提升至 70 至 85 tok/s，但這仍不是開箱即得的基準。
推論系統
The IOL-AI Challenge requires systems to infer the rules of an unfamiliar language from a small number of examples before completing translation, transcription, and correction tasks. Under a single-T4 constraint, the best submission raised the same model’s baseline GM score from 9.40 to 19.79, but still lagged well behind closed models that reached the gold-medal threshold.
推論系統
NVIDIA has open-sourced TensorRT Model Connect, which uses two commands to turn supported Hugging Face or local checkpoints into versioned bundles that can be handed off to C++ applications. It shortens the model-porting path, but remains a reference implementation; support and performance must be validated for each model, precision, and hardware configuration.
推論系統
Qualcomm AI Research adds per-layer fast weights to a frozen Transformer, absorbing context in 512-token segments so queries no longer need to read the source text. In 128K experiments, Qwen2.5-0.5B maintained fixed memory usage and outperformed ICL and RAG, though the evidence is currently limited to three synthetic retrieval tasks.
最新模型
TokEval incorporates numeric place value, program AST boundaries, UTF-8 integrity, and cross-lingual cost into tokenizer evaluation. Controlled experiments show that some metrics can predict specific downstream capabilities, although the results are currently limited to a single 1.27B-parameter architecture.
硬體
The open-source rl-triton library rewrites credit-assignment recurrences such as GAE, V-Trace, and Retrace as Triton associative scans, reducing sequential dependency depth from O(T) to O(log T). In tests with 4,096 environments and 128 steps, it was 1.6× to 5.70× faster than an already vectorized torch.compile baseline, although end-to-end PPO speedup for large policies was only about 1.02×.
推論系統
Rather than removing token connections, TileMix selects either the FP16 or INT8 Tensor Core path for each attention-score tile within a single fused kernel. It reaches 31.80K token/s on a 4K LLaMA 3.2 3B benchmark, though the results are currently limited to A100, static routing, and the prefill stage.
推論系統
Rather than expanding ultra-low-bit weights back into FP16, FluxBin uses binary patterns to directly index precomputed partial sums. The paper reports up to a 5.92× speedup and a 10.19× improvement in energy efficiency on a single A100, but its public code link remains inaccessible.
最新模型
Starting from Qwen3-4B, PertMind combines gene responses, pathway direction, and output formatting into computable rewards, eliminating the need to manually author large numbers of biological reasoning traces. The model transfers zero-shot to inverse perturbation identification and dual-perturbation tasks, but its natural-language mechanistic explanations still cannot be treated as causal proof.
推論系統
The new evaluation no longer provides a separate allowance for each problem. Instead, models must allocate tokens or tool calls across six problems. Even though all six models can solve the problems individually, under a shared budget they generally fail to direct compute toward the problems with the highest expected payoff.