Linus Torvalds <torvalds@xxxxxxxx> wrote: > On Wed, 18 Oct 2006, Shawn Pearce wrote: > > > > I guess its my turn then to work in the mmap window code, huh? :-) > > There are bigger reasons to _never_ allow packs to contain deltas to > outside of themselves: > > - there's no point. > - it's a bad design. That and all of the other reasons you cited in your message are why I haven't finished trying to use some sort of dictionary based compression for packing objects. On the other hand we've already seen how packs >1.5 GiB in size (certainly well within the 4 GiB limitation in the current index file format) cannot be repacked by git-repack-objects on a 32 bit address space as the entire pack file is mmap'd on one shot. After the kernel space of ~1 GiB and the pack file at ~1.5 GiB there's very little address space left for the application code. My comment that you quoted was about mmap'ing the pack files in large chunks (around 64-128 MiB at a time, but configurable from .git/config) rather than as an entire massive mapping. It had absolutely nothing to do about changing the pack file format, the index format, or any other on disk format. Although it would add a new pair of configuration options to .git/config. Is that change too radical? :-) With such a change the Git and Linux kernel repositories would both still mmap in one chunk but much larger projects like Mozilla or very large pack files coming out of git-fastimport would actually be usable on 32 bit architectures without running into address space limitations so quickly. Git would also be slightly more usable for some people who have a lot of very uncompressable data stored in Git. Unless of course you are actively working on a fix for the Linux kernel so that we can actually have all 4 GiB of virtual address space available for the userspace git-repack-objects process. Or have some sort of secret plan to upgrade everyone who uses Git to 64 bit processors which support 64 bit address spaces... -- Shawn. - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html