GitHub Repo
Paddock Open-Sources Rust/CUDA Inference Engine Targeting Concurrent AI Agents on a Single NVIDIA GPU
Truespar has released Paddock’s scheduler, paged KV cache, memory management, and custom CUDA kernels, along with OpenAI- and Anthropic-compatible interfaces. The authors’ benchmarks show it outperforming vLLM and SGLang on some single-GPU workloads, but the results have not been independently reproduced, and Paddock does not support tensor parallelism, ROCm, or Metal.