On 12/7/07, Nicolas Pitre <nico@xxxxxxx> wrote: > On Fri, 7 Dec 2007, Jon Smirl wrote: > > > Using this config: > > [pack] > > threads = 4 > > deltacachesize = 256M > > deltacachelimit = 0 > > Since you have a different result according to the source pack used then > those cache settings, even if there was a bug with them, are not > significant. > > > And the 330MB gcc pack for input > > git repack -a -d -f --depth=250 --window=250 > > > > complete seconds RAM > > 10% 47 1GB > > 20% 29 1Gb > > 30% 24 1Gb > > 40% 18 1GB > > 50% 110 1.2GB > > 60% 85 1.4GB > > 70% 195 1.5GB > > 80% 186 2.5GB > > 90% 489 3.8GB > > 95% 800 4.8GB > > I killed it because it started swapping > > > > The mmaps are only about 400MB in this case. > > At the end the git process had 4.4GB of physical RAM allocated. > > That's really bad. > > > Starting from a highly compressed pack greatly aggravates the problem. > > That is really interesting though. > > > Starting with a 2GB pack of the same data my process size only grew to > > 3GB with 2GB of mmaps. > > Which is quite reasonable, even if the same issue might still be there. > > So the problem seems to be related to the pack access code and not the > repack code. And it must have something to do with the number of deltas > being replayed. And because the repack is attempting delta compression > roughly from newest to oldest, and because old objects are typically in > a deeper delta chain, then this might explain the logarithmic slowdown. > > So something must be wrong with the delta cache in sha1_file.c somehow. I applied the delta accounting patch. It took about 200MB of from the memory use but that doesn't make a dent in 4GB of allocations. -- Jon Smirl jonsmirl@xxxxxxxxx - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
