Re: git clone dies (large git repository)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



"Troy Telford" <ttelford@xxxxxxxxxxxxxxxx> writes:

> I'm thinking of it as an option for git-repack-- that the user can set
> the maximum size of any individual pack, and after that limit is
> reached, a  new pack file is started.  (ie. --max-size 2GB) and will
> end up with two  packs, each 2GB in size.

The way I would suggest you do it is not by size but by distance
from the latest.  If you want to split the kernel history for
example, you repack up to 2.6.14 for example, and then repack
the remainder.  That way, you can optimize for size for older
(presumably less frequently used) data while optimizing for
speed for more reent stuff.

There is no wrapper support for the above splitting in
git-repack.  The low-level plumbing tools can be used this way
for example:

	name=`
                git rev-list --objects $list_old_tags_here |
                git pack-objects --window=50 --depth=50 --non-empty .tmp-pack
        ` &&
        mv -f .tmp-pack-$name.{pack,idx} .git/objects/pack/

	name=`
		git revlist --objects --all --not $list_old_tags_here |
		git pack-objects --non-empty .tmp-pack
	` &&
        mv -f .tmp-pack-$name.{pack,idx} .git/objects/pack/

If you are splitting into more than two, you would instead have
more than one $list_old_tags_here list, and iterate them
through, something like:

	pack_between () {
        	already_done="$1"
                do_this_time="$2"
                w=${3-10}
		name=`
			git rev-list --objects \
                        	$do_this_time \
                                --not $already_done |
			git pack-objects --window=$w --depth=$w \
                        	--non-empty .tmp-pack
                ` &&
                mv -f .tmp-pack-$name.{pack,idx} .git/objects/pack/
	}

	pack_between "" "$prehistoric_tag_list" 100
	pack_between "$prehistoric_tag_list" "$more_recent_tag_list" 50
	pack_between "$more_recent_tag_list" --all

All untested, of course, so do not play with it in your precious
repository you do not have any other copy, but hopefully you get
an idea ;-).

-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]