FlashVector Expands Performance Agents From GPU Kernels to the Entire Recommendation Model Serving Stack

Unity’s research team unveiled FlashVector on September 15, seeking to expand the coding agents used in recent years to optimize CUDA and Triton kernels into performance engineering agents capable of handling complete model serving systems. It targets the layered bottlenecks common in recommendation and advertising models: above the GPU kernels sit PyTorch computation graphs, NVIDIA Triton Inference Server, batching and request handling, and real-time feature transformation services. Accelerating only one layer often shifts the bottleneck elsewhere. [Paper](https://arxiv.org/abs/2609.17391)

FlashVector packages different technology stacks into extensible optimization environments, allowing an agent to read code, make changes, run benchmarks, and iteratively search using performance feedback. The cases described in the paper cover not only GPU computation, but also Triton’s C++ server code and feature transformation services written in Python. The key idea is not for a single model to possess all the necessary systems knowledge, but to provide suitable build, test, and measurement interfaces for each layer, enabling the same optimization loop to work across languages and components.

The team says the system has been deployed on the [Unity Vector advertising platform](https://unity.com/products/unity-ads/vector): model servers achieved up to 2× higher throughput and a 1.98× latency speedup, while feature storage services reached up to 1.6× higher throughput. The implication for engineering teams is that performance agents may be moving beyond “generating faster kernels” into automated systems engineering capable of modifying production services. Regression tests, SLOs, data correctness, and change reviews must therefore serve as hard gates beyond the reward function.

For now, the results still come primarily from a single advertising platform operated by the authors’ company, and only the maximum gains have been disclosed. The paper does not release the agent’s code, complete workloads, the proportion of unsuccessful modifications, or inference costs. Future scrutiny should focus on whether external teams can reproduce its changes, whether the performance gains hold across different hardware and traffic distributions, and whether the agent-generated low-level changes can withstand sustained production workloads.