Re: Performance issue: initial git clone causes massive repack

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Robin H. Johnson wrote:

> Hi,
> 
> This is a first in my series of mails over the next few days, on issues
> that we've run into planning a potential migration for Gentoo's
> repository into Git.
> 
> Our full repository conversion is large, even after tuning the
> repacking, the packed repository is between 1.4 and 1.6GiB. As of Feburary
> 4th, 2009, it contained 4886949 objects. It is not suitable for
> splitting into submodules either unfortunately - we have a lot of
> directory moves that would cause submodule bloat.
> 
> During an initial clone, I see that git-upload-pack invokes
> pack-objects, despite the ENTIRE repository already being packed - no
> loose objects whatsoever. git-upload-pack then seems to buffer in
> memory.
> 

Have you considered using a bundle as part of the initial clone process? The 
idea would be to periodically create a bundle

	git bundle create <somename>.bundle [list of refs]

and publish that on your website. A new user would then do

	wget $uri-of-bundle
	git clone <somename>.bundle
	cd $somename
	git remote add origin $origin
	git fetch

and they have the current repo. As the bundle is a file, it can be 
distributed by torrent or other method. The expense of creating the pack in 
the bundle is paid exactly once when the bundle is created.

Mark



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]