推論基礎設施
NVIDIA PAIR Routes Requests to Idle Computers Through a Compatible Proxy, Distributing Local Multi-Agent Inference Across Nodes
PAIR adds a compatible proxy layer between Ollama, LM Studio, and agents, routing independent requests to computers on the local network according to model availability and GPU load. It increases workload concurrency; it does not pool VRAM or split a single inference across multiple GPUs.
NVIDIA released a beta version of the open-source Personal AI Router (PAIR) on September 3, aiming to turn Windows, Linux, and macOS computers in homes or small studios into a flexible inference pool. PAIR is not a new inference engine; each node still relies on Ollama or LM Studio to load and run models. It takes over compatible local API endpoints, allowing existing agent frameworks to point to a single address without needing to know how many machines are behind it.
Nodes can discover one another automatically through mDNS or be added by IP address. After the user approves pairing, nodes communicate using mutual TLS and generated certificates. The scheduler checks whether each machine is online, the status of its inference engine, whether the requested model is present, the number of active jobs, and GPU utilization before assigning a new request to an eligible node. Streaming responses return through the same proxy. A node can leave the available pool when it is asleep, offline, or occupied by a foreground application.
The design primarily addresses queuing when multiple agents or local applications invoke models simultaneously, rather than insufficient capacity for a single model. Each request remains on the same node from start to finish, so PAIR does not perform tensor parallelism, model sharding, or cross-machine VRAM pooling. Supported hardware includes the RTX 20 Series and newer, RTX PRO, DGX Spark, and devices with Apple M4 or newer chips.
In NVIDIA’s demonstration using five sub-agents, Qwen 3.6 35B A3B, and Ollama, three heterogeneous devices completed the workload in an average of 8 minutes 48 seconds, compared with 18 minutes on a single RTX Spark laptop. This is only a demonstration of a specific configuration; without workload-distribution details, network-latency measurements, or repeated multi-run data, it cannot be interpreted as evidence of linear speedup. Engineering teams should next examine retry semantics when a node fails mid-request, version consistency among identically named models, scheduling fairness, and whether mDNS, certificate rotation, and proxy endpoints can withstand the threat model of an untrusted local network.