OpenAI details how its Habitat storage platform supports over 1 billion ChatGPT users

OpenAI releases the second part of its Habitat storage platform series, detailing how it scaled Azure Cosmos DB to serve over 1 billion ChatGPT users. The platform hosts more than 500 PB of data across 40 geographic regions. It has cycled through shared multi-tenant architectures, single-tenant containers, and back to a default multi-tenant setup. Working with Azure, it addressed noisiness and weak partitioning issues using hot-key rate limiting, QoS tiering, and an updated serverless mode.