推論系統／本機 AI
NVIDIA PAIR Dispatches Cross-Device Inference Through a Single Local Endpoint, but Does Not Pool GPU Memory
The open-source Personal AI Router can schedule independent Ollama or LM Studio requests across Windows, Linux, and macOS nodes. It is suited to increasing multi-agent concurrency, but it is not a distributed model execution engine: a single model must still fit entirely on one machine.
NVIDIA has released a beta of the Apache 2.0-licensed Personal AI Router (PAIR), aiming to turn RTX PCs, DGX Spark systems, and Apple Silicon Macs on the same local network into a shared local inference pool. Applications need only connect to a single Ollama-compatible or OpenAI-compatible endpoint. PAIR tracks node availability, inference engine status, whether models are loaded, and GPU utilization, then routes each independent request to an appropriate machine. As a result, existing agent frameworks generally require no modification.
According to [NVIDIA’s technical overview](https://developer.nvidia.com/blog/nvidia-pair-virtual-inference-router-expands-available-compute-on-your-local-network/), nodes discover one another via mDNS, and traffic is protected by mutual TLS after pairing. The system currently supports Windows 11, Linux, macOS, x64, and Arm64, while inference itself is handled by Ollama or LM Studio. In NVIDIA’s demonstration, Hermes Desktop created five sub-agents and completed the workload across three devices in 8 minutes 48 seconds, compared with 18 minutes on a single RTX Spark laptop. However, this is a vendor demonstration, not yet an independent benchmark covering different combinations of models, networks, and hardware.
The most important architectural boundary is that PAIR performs request-level scheduling only. The [project documentation](https://github.com/NVIDIA/Personal-AI-Router) explicitly states that each request is still handled from start to finish by a single node. PAIR does not pool VRAM, shard one model across multiple hosts, or split a sequence while it is being generated. If a 70B model cannot fit on any individual node, adding more small GPUs will not solve the capacity problem. The main benefits arise when multiple agents or sessions can issue mutually independent calls in parallel.
The claim that “data stays local” is also conditional: the client, model source, inference engine, and all nodes must remain local, while external cloud tools may still transmit data off-device. Engineering teams should next measure scheduling fairness across heterogeneous nodes, model cold-load costs, and retry behavior during streaming interruptions or node sleep. They should also include pairing certificates, proxy listening interfaces, and automatically created firewall rules in their threat models.