On Mon, 11 Feb 2008, Martin Koegler wrote: > On Sun, Feb 10, 2008 at 07:33:34PM -0500, Nicolas Pitre wrote: > > On Sun, 10 Feb 2008, Junio C Hamano wrote: > > > > > mkoegler@xxxxxxxxxxxxxxxxx (Martin Koegler) writes: > > > > > > > This patch adds a cache to keep the object data in memory. The delta > > > > resolving code must also search in the cache. > > > > > > I have to wonder what the memory pressure in real-life usage > > > will be like. > > > FWIW, I don't like this idea. > > > > I'm struggling to find ways to improve performances of > > pack-objects/index-pack with those large repositories that are becoming > > more common (i.e. GCC, OOO, Mozilla, etc.) Anything that increase > > memory usage isn't very welcome IMHO. > > Maybe I have missed something, but all repack problems reported on the > git mailing list happen durring the deltifing phase. The problematic > files are mostly bigger blobs. I'm aware of these problems, so my > patch does not keep any blobs in memory. This is not only repack, which is something that few people should run anyway. It is also index-pack which takes an awful amount of time on the OOO repo (much less than the repack but still), and that's something that is visible to anyone cloning it. So the comparison with pack-objects is useless. In absolute terms, index-pack has to inprove, not regress. > So my conclusion is, that the memory usage of index-pack with --strict > should not be too worse compared to pack-objects. Well, the comparison is flawed anyway. Agressively repacking the OOO repo is reported to take around 14 GB of RAM if not bounded, whereas index-pack remained around 300MB. But the buffer cache managed by the kernel is also a big performance factor and anything that requires more memory in user space will affect that cache. Then remember that this repo has 2411764 objects. > Please remember, that --strict is used for pushing data. And what exactly does it provide in terms of safetiness that index-pack doesn't already do? I'm not sure I fully understood the goal here. Nicolas - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
