GitHub's New Git Storage: How to Pick a Clone Strategy for Large Repos

GitHub published the details of its new Git storage architecture on November 5, 2026, reporting 5,000 concurrent agents and 7,200 pushes per second on a single repository in its internal testing. But if you run a large monorepo and your clones and CI checkouts are the slowest part of the developer loop, here is the finding that matters for your next decision: nothing in GitHub’s post claims your git clone gets faster. The architecture is a server-side change, its figures are vendor-internal tests rather than audited client benchmarks, and it sits in limited private preview today. So the clone-strategy question stays exactly where it was, answered by repo size, workflow shape, and tool tolerance. The independent evidence for that answer, including Uber’s GitFarm paper and an empirical study of GitHub Actions caching, is preprint-reported, based on the authors’ own measurements, and the guidance below carries those limits.

What GitHub actually announced

The engineering post, “Inside GitHub’s new Git storage architecture,” describes a rebuild of how GitHub stores and serves Git data, announced at GitHub Universe and developed under the name Project Rubicon, which began in May 2026 after GitHub’s initial Azure migration. The motivation is agent traffic: GitHub reports that daily requests to one top repository grew about 4x between March and May of this year and kept climbing, while the busiest repositories on GitHub.com today receive at most dozens of writes per second, typically in short bursts. The design target is at least two orders of magnitude more scale than that.

The mechanism has three pieces, all server-side:

• Pack data in Azure Blob Storage. Every push adds a pack to Blob Storage. Git can serve a repository from many packs, but each additional pack makes object lookups slightly more expensive, so pack count cannot grow without bound.
• Ref-CAS, a compare-and-swap service for references. It transacts reference state alone at tens of thousands of operations per second, never holds or validates object data, and virtual replicas upload a push’s pack to Blob Storage before committing the reference transaction.
• Background compaction. In the legacy architecture, compaction runs on the same replicas that serve traffic, on every replica. In the new one, it runs entirely in the background on separate nodes, directly against durable storage, using Ref-CAS to notify virtual replicas of the evolving state.

This is a write-path and storage-layout redesign that also reworks the read path: virtual replicas serve reads from local caches that can be rebuilt at any time, and Blob Storage holds read accelerators and cached responses for frequently repeated fetches. It addresses a world where thousands of automated agents push and read against one repository concurrently. It is in limited private preview now, with a broader rollout planned for early next year, and the preview’s terms and eligibility are not stated in the post.

The claim the post does not make

Read the post as a client and one thing stands out: there is no measurement of clone duration, fetch latency, or checkout time for any repository size on the new architecture. The 5,000-agent and 7,200-push figures are internal tests of server write capacity under concurrent read traffic.

GitHub’s own framing of the client side is an admission of unfinished work:

“Partial clone, shallow clone, and sparse checkout each let an agent fetch less. Each one has gaps. They don’t always work well together, either. We want to improve each piece and make them fit.”

It also concedes that Git still has code paths that slow down much faster than the repository grows. The practical consequence is direct. If your clones are slow today, the fix today is the same set of levers it was last month. The storage redesign may raise the ceiling on how large a repository GitHub will gracefully host, which lowers the cost of keeping a growing monorepo intact, but it shifts the performance burden to your clone and cache design rather than removing it.

Picking a clone strategy: the four options

The independent evidence base for this decision comes largely from Uber’s scale, so treat magnitudes with care. The GitFarm paper is a preprint describing one company’s multi-gigabyte monorepos and millions of Git invocations per day; its numbers may overstate costs for a typical repository. Its failure modes, though, generalize, because they are properties of Git itself.

Full clone remains the default for a reason: every tool works. Merge-base, bisect, blame, code review archaeology, release tagging, and any tool that walks history all assume complete objects. The cost is the full data volume on every clone, which is exactly what hurts in CI. If your tooling needs complete history and your repository is still a size where a clean clone takes seconds to a couple of minutes, full clone plus a decent CI cache is the lowest-risk configuration, and none of the cited evidence argues otherwise.

Partial clone with blob filters plus sparse checkout is the standard large-monorepo answer: fetch commits and trees eagerly, blobs on demand, and materialize only the directories you work in. For interactive developer machines this is usually the right call once clones cross into minutes. The catch is documented and sharp: the GitFarm authors report that “partial clones frequently break tooling assumptions in large monorepos,” and that while such techniques can reduce initial data transfer, they “do not eliminate server-side computation during clone and fetch operations, and often impose functional limitations.” Before recommending it, audit the toolchain your repo actually runs: any script that assumes every blob is local will either fail or trigger a storm of on-demand fetches that can be slower than the clone you avoided.

Shallow clone is the right answer only for jobs that never need history. The same preprint states that “shallow history restricts operations such as merge-base and bisect.” That rules shallow clones out for merge queues, changelog generation, semantic-release tooling, and anything that diffs against a base branch it must discover rather than is told. It rules them in for build-and-test jobs on a known commit, deployment packaging, and agents that only read a snapshot.

CI mirror caching and checkout caching attacks fetch amplification rather than per-clone size. Here the counter-evidence is strongest. The GitFarm paper reports that CI platforms like Jenkins and Buildkite reuse local mirrors, “yet per-agent caches quickly diverge and require expensive fetches on CI job execution. Even with caching, large monorepos often spend several minutes preparing before execution.” It describes the broader pattern as “significant infrastructure overhead, manual maintenance, inconsistent cache hit rates, and cold start latencies of several minutes for large monorepos,” with thousands of independent clone and fetch operations loading upstream Git servers. In other words, the cache you added to fix slow clones can become a second system to maintain, with hit rates that decay as agents and jobs churn.

A decision table condenses this:

One honest gap in this table: no cited source provides clone-time measurements per repository size on GitHub’s new storage, so the “best fit” column rests on pre-rollout evidence from other environments, not on benchmarks of the architecture this article is about.

The caching treadmill, measured

If your instinct is to lean on GitHub Actions caching to paper over slow checkouts, the empirical picture is cautionary. A study of how developers adopt and evolve CI caching on GitHub Actions documents the mechanics: caching is typically configured as a dedicated step with two user-specified parameters, path and key, and behavior varies by setup action, with actions/setup-go@v4 enabling caching by default where earlier versions do not. More interesting is the churn finding. The authors report “highly frequent cache maintenance, particularly repeated parameter updates in build and test jobs that occur every few days,” while cautioning that “it remains unclear whether such repeated adjustments translate into actual performance gains.”

That combination, constant retuning with unproven payoff, is a maintenance treadmill. It does not mean caching is useless. It means that for checkout data specifically, you should measure hit rate and cold-start time before and after adding a cache layer, and you should treat recurring cache-key surgery as a cost line, not a free optimization.

The endpoint alternative: executing Git remotely

Uber’s answer to the same problem was to stop cloning at all. GitFarm moves Git operations to a remote, repository-centric service, and the preprint reports production results from a rollout across Uber’s developer platform since early 2025: client service initialization latency cut from up to 15 minutes to less than a minute, and client-side resource utilization lowered by more than 70%. Those are author-reported figures from a single preprint, and they describe Uber-scale monorepos.

The transferable idea is narrower than the numbers: when per-job fetch amplification dominates, meaning thousands of CI jobs and agents each independently pulling overlapping data from the upstream, consolidating Git execution behind a shared, warm service beats optimizing each client in isolation. You do not need to build GitFarm to apply this. A shared, persistent mirror or a repository-close runner pool is the same shape of solution at smaller scale, with the same tradeoff: you are now operating infrastructure whose failure takes down every job at once. I would trial this pattern only where measured clone-and-fetch time across CI, not a single slow clone, justifies the operational surface.

If you benchmark the rollout, do not fool yourself

Once the broader rollout ships early next year, many teams will re-time their clones to see whether anything changed. One documented case argues for pinning your measurement environment first. The MooBench performance-change detection work describes a regression on GitHub-hosted runners that, per its abstract, traced to a Linux kernel version change rather than to any software change. The regression appeared between two benchmark runs whose commits installed an R package that was pre-installed in the ubuntu-22.04 image and obsoleted in ubuntu-24.04, “a decision by GitHub”; the paper reports that image change as the surrounding context, and its authors state that future work will validate whether the regression is related to different kernel versions. If you compare clone times before and after the storage rollout on hosted runners without pinning the image, you may measure an image change and credit the storage layer. Pin the runner image, keep the repository content as stable as you can, run enough repetitions to see through variance, and treat any single before-and-after number as unverified until it survives those controls.

The decision, restated with its limits

For now, treat GitHub’s new Git storage as a server-side change with no demonstrated client-side clone speedup, and pick your clone strategy from the decision table on repo size, workflow shape, and tool tolerance. Demand measured hit rates before and after any cache work, and re-measure once the broader rollout ships, with pinned runner images.

The limits are the limits of the evidence: vendor-internal tests of a preview system, and preprint-reported numbers from Uber-scale monorepos and observational CI-configuration data that may not transfer to your repository. The sources cited here do not establish that the new storage fails to improve clone times; no independent measurement of clone time on the new architecture appears in them. If GitHub’s stated plan to make partial clone, shallow clone, and sparse checkout “fit” together ships, the tradeoffs in the table above are exactly what to revisit first.

Frequently Asked Questions

Does GitHub’s new Git storage make git clone faster?

nothing in GitHub’s post claims your git clone gets faster. The architecture is a server-side change, its figures are vendor-internal tests rather than audited client benchmarks, and it sits in limited private preview today.

When is GitHub’s new Git storage architecture available?

It is in limited private preview now, with a broader rollout planned for early next year, and the preview’s terms and eligibility are not stated in the post.