OpenAI Rewrites Habitat from Python in Rust, Improving CPU and Memory Efficiency by 6× and 15×

OpenAI has unveiled Habitat, the online storage platform that supports ChatGPT, Codex, the API, and internal services. It currently spans nearly 40 regions, processes more than 70 million requests per second, and serves over 500 PB of data. However, it is not a new model storage format. Rather, it is a unified data access layer positioned between products and resources such as Azure Cosmos DB, Valkey, and Blob Storage, centralizing schema management, routing, authorization, encryption, multi-tenant isolation, rate limiting, and change data capture.

Habitat began as a Python client library embedded in applications. As the number of services using it grew, coordinating version and protocol changes gradually became a bottleneck, prompting OpenAI to separate it into a standalone service in 2025. The Python version handled more than 20 million requests per second at one point, but CPU-intensive routing, compression, and encryption workloads could block the `asyncio` event loop, amplifying tail latency. The team mitigated the issue by monitoring event-loop lag, limiting per-process concurrency, and adding more workers. CPU spikes caused by synchronous parsing of feature flags were spread out by reducing configuration size, extending refresh intervals, and adding jitter.

In the second quarter of 2026, two engineers, working with Codex and GPT-5.5, rewrote the service in Rust. The new version now handles 95% of production requests. According to OpenAI’s measurements, it is 6× more CPU-efficient and 15× more memory-efficient than the Python version, while also reducing average and tail latency. The lesson is not that “Rust will inevitably replace Python,” but that teams can first use a language that enables faster iteration to clarify service boundaries, then rewrite hot paths once core count and memory become the dominant costs.

Engineering teams should still interpret these multipliers cautiously. OpenAI has not disclosed the hardware, workload distribution, latency percentiles, or a complete feature-parity comparison between the versions before and after the rewrite, so external parties cannot reproduce the results. Habitat also deliberately routes complex search and analytics queries to Rockset to avoid burdening the OLTP path, meaning the gains come partly from workload isolation rather than from the programming language alone. A follow-up article is expected to cover Cosmos DB, multi-tenant reliability, and read caching—details needed to assess how the overall architecture achieves its current scale.