GitHub rebuilt its git locks for agents, three to four times throughput

GitHub engineers Eleanor White and Victoria Dye published a long post describing how the company rebuilt the storage layer under its git repositories so that fleets of coding agents can push into the same tree without stalling each other. The post sits on The GitHub Blog under engineering and architecture optimization.

The traffic that forced the change

GitHub's internal Spokes API handles more than 100 billion git operations a day. A typical repository sees 100 to 1,000 reads for every write. The old design locked a whole repository for the length of a write, so two writes to two different branches had to queue behind each other even when they touched nothing in common.

The move is to per-reference quorum serialisation. The server now orders writes at the level of a single reference, which is usually one branch or one tag. Two agents pushing to two different branches can commit at the same time. The authors state the consistency rule plainly: "Git's consistency requirements apply to each reference individually. Once the server accepts an update to a reference, no client should see an older value afterward."

Virtual replicas and the Azure migration

The second change is virtual replicas on top of Azure Blob Storage. Short-lived compute nodes bridge the write quorum to blob storage and act as replicas without the cost of a full replica machine. GitHub says 99.99999 percent of repositories are now replicated in Azure, with the last 0.00001 percent held back by a 25 ms round-trip to the Azure Central US region, 1,000 miles from the data centre.

Read and write paths are decoupled too, so each can scale on its own.

What the numbers show

The authors report three to four times higher throughput at lower latency on per-reference serialisation in internal tests. The post does not state before and after numbers for Azure Blob and the virtual replicas, so the throughput figure applies only to the lock change.

The reason for all three changes is the same. A team of humans writing into one repository sends a handful of writes an hour. A fleet of agents sends one every few seconds for days, often from the same session. The post names agent traffic as the shape that paid for the rewrite, with ordinary scaling as a secondary benefit.

For any team building a code host that agents will push into, or scoping how their own git service will hold up when the ratio of agent to human commits reverses, GitHub's post is a primary account with its own numbers attached.

Source

The GitHub Blog: The bridge to agent-scale git infrastructure, by Eleanor White and Victoria Dye.