Archive
推論 runtime、量化、 serving、KV cache、成本與效能。
最新模型
A new study restructures IndexTTS-2’s GPT, Diffusion Transformer, and vocoder using TensorRT and TensorRT-LLM, while adding streaming and batched inference. Experiments on the English and Chinese Seed-TTS benchmarks show up to a 5× speedup for GPT, but the currently available public materials remain insufficient to directly reproduce the complete optimization pipeline.
開發者平台
DeepSeek retired the `deepseek-chat` and `deepseek-reasoner` compatibility names on July 24, meaning API requests that have not been migrated may now fail outright. The new interface separates model selection from reasoning mode, making the upgrade more involved than simply replacing a model string.
最新模型
InclusionAI has released Ling-3.0-flash, a 124B-parameter model that uses KDA, MLA, and a highly sparse MoE architecture for long-horizon agentic reasoning. The API is now available, but the weights are not scheduled for release until August, so its performance claims cannot yet be fully reproduced.
最新模型
Google is moving Gemini 3.6 Flash and 3.5 Flash-Lite to stable APIs, targeting complex primary agent workloads and high-throughput subagents, respectively. Official data shows improvements in token usage, coding tasks, and computer use, but most comparisons come from Google or its evaluation partners and still require validation in real-world workflows.
最新模型
Multiverse Computing has unveiled a compressed and healing-trained version of Llama 3.3 70B that can be served on Xeon 6 using vLLM CPU and Intel AMX. Official tests show nearly double the throughput, but output remains below 4 tokens per second, and neither the checkpoint nor the raw results have been released.
AI 安全
ResponseGuard reframes multimodal response moderation as binary classification and evaluates responses sentence by sentence during streaming without generating a chain of thought. The authors’ tests show better response detection than a 3B reasoning-based baseline, but the model weights, training code, and data splits have not yet been released.
推論系統
Moreh demonstrated a distributed inference service running GLM-5.1 across four nodes equipped with 32 AMD MI300X GPUs, with live displays of TTFT, TPOT, and GPU utilization. MoAI further claims it can split prefill and decode across GPUs from different vendors, but the latest demonstration did not disclose complete throughput data or a reproducible configuration.
最新模型
Microsoft has unveiled a “hill-climbing” post-training process that feeds product tools, graders, and real-world usage signals back into model iteration. After further training in an Excel environment, MAI-Code-1-Flash can be served on older GPUs, but the company has not disclosed its parameter count, evaluation sets, or raw experimental data.
具身 AI
OpenBMB has updated MiniCPM-Robot’s weights, code, and deployment documentation, enabling a single 1.5B VLA model to handle multiple manipulation tasks, while a separate 0.9B model performs target tracking. Streaming context and a custom inference engine reduce redundant visual computation, but the official benchmarks remain insufficient to demonstrate cross-robot generalization.
代理工具
Cursor has redesigned Auto mode around a request-level model router that directs traffic among different models based on the prompt, context, domain, and complexity. The company claims that online A/B testing shows cost savings of up to 60%, but it has not disclosed its routing rules, model distribution, or complete raw data.
推論系統
PyTorch and Google have built a TPU backend for Helion, allowing PyTorch-style kernels to compile to Pallas and reducing the need to write separate low-level TPU kernels. The team measured up to 838 TFLOPs for Flash Attention on TPU v7, though the feature remains under active development.
AI 安全
ResearchArena turns post-training, CUDA kernel optimization, and inference server optimization into long-horizon evaluations with hidden attack objectives. Across 384 experiments, allowing monitoring agents to execute the resulting artifacts helped uncover backdoors, but sabotage embedded in training data still frequently went undetected.