Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx> wrote: > On Wed, 28 Mar 2007, Dana How wrote: > > Of course this is unusable, since object_entry's in an .idx > > file have only 32 bits in their offset fields. I conclude that > > for such large projects, git-repack/git-pack-objects would need > > new options to control maximum packfile size. > > Either that, or update the index file format. I think that your approach > of having a size limiter is actually the *better* one, though. Nico and I were hoping we could push the index file format change back until pack v4 was also worthy of merging. So I had also started work on an index-pack splitter: URL: git://repo.or.cz/git/fastimport.git Branch: sp/splitpack Its far from complete. > Well, it does affect fetching, in that "git index-pack" obviously would > also need to be taught how to split the resulting indexed packs up into > multiple smaller ones from one large incoming one. But that shouldn't be > fundamentally hard either, apart from the inconvenience of having to > rewrite the object count in the pack headers.. We already do this if its a thin-pack that is being made non-thin. So its annoying, yea, but we already have one toe in that particular pond... > To avoid that issue, it may be that it's actually better to split things > up at pack-generation time *even* for the case of --stdout This is actually a pretty good idea. index-pack knows when the end of its current packfile is; if there are 12 more bytes available in the input stream and it looks like another pack header then it can just restart itself over again with that next pack. This is actually a pretty small change in index-pack, certainly a lot less intrusive than what I was trying to do in my sp/splitpack topic above. -- Shawn. - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html