All News

1537 technical news articles

推論系統

KVDiagnosis Traces KV-Cache Compression Failures Sample by Sample, Releases 12,520 Correct-to-Wrong Cases

Rather than merely comparing aggregate long-context scores, KVDiagnosis creates a paired FullCache baseline for every compression configuration to track evidence retention, attention, and probability drift. An implementation audit also led the team to exclude 7,800 PyramidKV records with identical results, highlighting how adapter errors can contaminate cross-method evaluations.

推論系統

Laguna S 2.1 INT4 With DFlash: Four RTX 3090s Tested at 200K Context and a Peak 282.5 tokens/s

A publicly available, reproducible community vLLM recipe deploys Laguna S 2.1 INT4 on four 24GB RTX 3090 GPUs and handles a real 190K-token request using a precision-matched DFlash draft model. The reported 282.5 tokens/s is a peak under a specific configuration, not sustained throughput, while the quality and compatibility of low-bit quantization remain highly dependent on the checkpoint and inference engine.

推論系統

CoBa Dynamically Allocates Inference Compute, Matching Best-of-16 With 58.9% Fewer Weighted Tokens

CoBa initially generates a small number of candidate answers, then uses disagreement and verification scores to decide whether to sample additional candidates or activate a stronger verifier. It achieved 85.13% macro-average accuracy across five reasoning benchmarks, although the results came from offline replay of pre-generated candidate pools rather than testing in a live service.