There have been several suggestions to improve Git clone speeds and reliability by supplementing the Git protocol with static content. The Packfile URI [0] feature lets the Git response include URIs that point to packfiles that the client must download to complete the request. Last year, Ævar suggested using bundles instead of packfiles [1] [2]. This design has the same benefits to the packfile URI feature because it offloads most object downloads to static content fetches. The main advantage over packfile URIs is that the remote Git server does not need to know what is in those bundles. The Git client tells the server what it downloaded during the fetch negotiation afterwards. This includes any chance that the client did not have access to those bundles or otherwise failed to access them. I agreed that this was a much more desirable way to serve static content, but had concerns about the flexibility of that design [3]. I have not heard more on the topic since October, so I started investigating this idea myself in December, resulting in this RFC. I focused on maximizing flexibility for the service that organizes and serves bundles. This includes: * Bundle URIs work for full and partial clones. * Bundle URIs can assist with git fetch in addition to git clone. * Users can set up bundle servers independent of the remote Git server if they specify the bundle URI via a --bundle-uri argument. This series is based on the recently-submitted series that adds object filters to bundles [4]. There is a slight adjacent-line-add conflict with js/apply-partial-clone-filters-recursively, but that is in the last few patches, so it will be easy to rebase by the time we have a fully-reviewable patch series for those steps. The general breakdown is as follows: * Patch 1 adds documentation for the feature in its entirety. * Patches 2-14 add the ability to run ‘git clone --bundle-uri=’ * Patches 15-17 add bundle fetches to ‘git fetch’ calls * Patches 18-25 add a new ‘features’ capability that allows a server to advertise bundle URIs (and in the future, other features). I consider the patches in their current form to be “RFC quality”. There are multiple places where tests are missing or special cases are not checked. The goal for this RFC is to seek feedback on the high-level ideas before committing to the deep work of creating mergeable patches. Testing with this series ======================== To get a full test of the features being proposed here, I created an MVP of a bundle server by pushing bundle data to some publicly-readable Azure Storage accounts. These bundle servers mirror the following public repositories on GitHub: * git/git * git-for-windows/git * homebrew/homebrew-core * cocoapods/specs * torvalds/linux In addition, the Azure Storage accounts are available in different regions: * East US: https://gitbundleserver.z13.web.core.windows.net * West US: https://gitbundleserverwestus.z22.web.core.windows.net * Europe: https://gitbundleservereurope.z6.web.core.windows.net * East Asia: https://gitbundleservereastasia.z7.web.core.windows.net * South Asia: https://gitbundleserversouthasia.z23.web.core.windows.net * Australia: https://gitbundleserveraustralia.z26.web.core.windows.net To test this RFC against these servers, choose your $org/$repo to clone and your region's bundle server $url and run $ git clone --bundle-uri=$url/$org/$repo/ https://github.com/$org/$repo Note that these servers are set up using "almost free" storage in Azure. Network connectivity of this storage can be slower than that of GitHub data centers, so your results may vary.
