AI coding infrastructure
GitHub Uses Agents to Rewrite the Copilot Runtime as 830,000 Lines of Rust, Eliminating Per-Client Node.js Overhead
Copilot’s shared agent runtime has migrated from TypeScript/Node.js to native Rust and can be embedded directly into SDKs for six languages through a C ABI. GitHub’s stress tests showed session throughput rising to 120 per second, though the figures come from internal tests that excluded model and network latency.
GitHub has published a complete account of the Copilot agent runtime migration: over roughly fourteen and a half weeks, one primary engineer used the Copilot app, CLI, and numerous sub-agents to rewrite an actively evolving TypeScript system as 832,378 lines of production Rust through 128 incremental PRs. The project also retained about 469,000 lines of Rust unit tests and 175,000 lines of end-to-end TypeScript tests. During that period, 135 versions were released from the main branch, avoiding a single big-bang cutover.
Under the old architecture, each language SDK launched a CLI subprocess and called the agent loop running in Node.js/V8 over JSON-RPC. This added one process per client, interprocess I/O, and a minimum working set of roughly 100 MB. The new design separates the TUI from the runtime. A native library exchanges JSON-RPC through around a dozen fixed C ABI entry points and can be loaded in-process by TypeScript, Python, Go, C#, Java, and Rust, while retaining an out-of-process mode. Public SDK changelogs confirm that Node.js, Python, Go, and Rust initially received experimental FFI transports, while Java also adopted platform-specific native packages.
GitHub excluded model inference and network time by using a local deterministic chat server with fixed, short responses. In a lifecycle test of 1,000 single-turn sessions, the legacy implementation completed 7.55 sessions per second, the out-of-process Rust version completed 57.45, and the in-process version completed 120. The incremental private resident memory for ten clients fell from 1,383 MB to 126 MB. These figures measure service-framework overhead, not actual Copilot response speed, and other changes were made during the same period, so the gains cannot be attributed solely to the programming language.
Of greater interest to engineering teams is the validation methodology. Agents generated most of the code, consuming about 136.3 billion tokens in total and incurring a nominal model cost of roughly $120,000. Yet as of September 14, dozens of merged, compiling regressions had still been found, including serialization, event-ordering, and lifecycle errors. GitHub therefore argues that the same agent implementing a rewrite must not be allowed to relax tests, compatibility baselines, or exemption labels on its own. The next areas to watch are native binary packaging in SDK v2, cross-platform version pinning, and the real-world multi-tenant benefits when model latency is included.