# Inside GitHub’s New Git Storage Architecture GitHub unveils Rubicon, a scalable storage architecture designed to support massive agentic development workloads. - **Source:** [GitHub Engineering](https://github.blog/category/engineering) - **Published:** 2026-11-05 - **Topic:** [technology](https://brifing.si/c/technology) - **Original article:** - **On Brifing:** https://brifing.si/s/bc41990a-6925-41e2-9432-fea69f13f89f ## New Storage Architecture GitHub recently introduced a new Git storage architecture designed to handle thousands of writes per second. This system supports 5,000 concurrent agents working on a single repository while maintaining low-latency read performance. The infrastructure is built to scale independently, ensuring that GitHub can support the growing demands of automated agentic development. ## Core Design Principles The architecture relies on two fundamental principles: minimizing coordination and decoupling storage from compute. By separating these concerns, GitHub allows different components to scale independently. This design shift ensures that Git operations remain familiar to developers while providing the backend robustness required for high-concurrency environments and massive repository traffic. ## The Rubicon Project Launched in May 2026, Project Rubicon serves as the foundation for this new storage model. It utilizes Azure Blob Storage for durable data and a specialized service called Ref-CAS for reference updates. This approach allows GitHub to maximize throughput while maintaining strict correctness, durability, and availability guarantees for all users. ## Handling Horizontal Scale GitHub maintains horizontal scale by partitioning tenants across infrastructure and enforcing traffic fairness. Spokes continues to handle routing and load balancing, ensuring that requests are distributed effectively. These battle-tested methods allow the platform to manage many developers and agents working across millions of repositories without compromising overall system performance or stability. ## Vertical Scaling Challenges Vertical scale focuses on how many agents can operate on a single repository simultaneously. As agentic workflows increase, GitHub expects sustained high-volume bursts rather than short-lived spikes. The new architecture targets two orders of magnitude more scale, ensuring the platform remains performant as individual repository sizes and activity levels grow exponentially. ## Virtual Replica Fleet Spokes routes requests to virtual replicas, which are disposable compute nodes that serve reads and writes. These replicas use local disk caches to keep latency low. Because correctness does not depend on a specific replica, the system can dynamically add more nodes as repository activity increases, ensuring consistent performance during peak usage. ## Ref-CAS Service Role Ref-CAS, or Reference Compare-and-Swap, is a replicated log service dedicated to managing reference state. Unlike legacy systems, it does not store or validate object data, only pointers. This allows the system to process tens of thousands of reference updates per second, providing a high-throughput baseline for repository versioning and synchronization. ## Optimizing Write Throughput To achieve high throughput, GitHub moved away from using Blob Storage for reference state. While object storage is excellent for durable data, it struggles with high-frequency reference updates. By implementing a dedicated consensus-based log service, the team successfully eliminated the write bottlenecks that limited previous prototypes to only mid-hundreds of operations. ## Decoupled Compaction Nodes In the new architecture, compaction runs entirely in the background on separate nodes. These nodes work directly against Blob Storage and Ref-CAS to merge packs and remove unreachable objects. This separation ensures that virtual replicas can dedicate their CPU and memory resources exclusively to serving incoming read and write requests. ## Read and Write Flow When a write occurs, a virtual replica indexes the pack, uploads it to Blob Storage, and commits a transaction to Ref-CAS. For reads, the replica checks the version against Ref-CAS and pulls necessary packs into its cache. This workflow ensures that reads remain cheap and fast even when repository state evolves. ## System Fault Tolerance Spreading write operations across multiple nodes improves fault tolerance and speeds up recovery during node failures. Because indexing and object connectivity checks are CPU-intensive, distributing this work prevents any single node from becoming compute-bound. This modular approach allows GitHub to maintain high availability even under extreme, sustained traffic from automated agents. ## Future of Development This architecture represents a significant shift toward supporting the future of agentic development. By planning for exponential growth in traffic and read fan-out, GitHub ensures its infrastructure will not become a bottleneck. The transition to this decoupled, high-throughput model provides the necessary foundation for the next generation of software engineering tools. ## Sources covering this story - https://github.blog/engineering/architecture-optimization/inside-githubs-new-git-storage-architecture/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._