On Fri, 7 Dec 2007, Jon Smirl wrote: > On 12/7/07, Nicolas Pitre <nico@xxxxxxx> wrote: > > On Fri, 7 Dec 2007, Jon Smirl wrote: > > > > > git repack -a -d -f --depth=250 --window=250 > > > > > > complete seconds RAM > > > 10% 47 1GB > > > 20% 29 1Gb > > > 30% 24 1Gb > > > 40% 18 1GB > > > 50% 110 1.2GB > > > 60% 85 1.4GB > > > 70% 195 1.5GB > > > 80% 186 2.5GB > > > 90% 489 3.8GB > > > 95% 800 4.8GB > > > I killed it because it started swapping > > > > > > The mmaps are only about 400MB in this case. > > > At the end the git process had 4.4GB of physical RAM allocated. > > > > That's really bad. > > > > > Starting from a highly compressed pack greatly aggravates the problem. > > > > That is really interesting though. > > > > > Starting with a 2GB pack of the same data my process size only grew to > > > 3GB with 2GB of mmaps. > > > > Which is quite reasonable, even if the same issue might still be there. > > > > So the problem seems to be related to the pack access code and not the > > repack code. And it must have something to do with the number of deltas > > being replayed. And because the repack is attempting delta compression > > roughly from newest to oldest, and because old objects are typically in > > a deeper delta chain, then this might explain the logarithmic slowdown. > > > > So something must be wrong with the delta cache in sha1_file.c somehow. Staring at the cache code I don't see anything wrong with it. > I applied the delta accounting patch. It took about 200MB of from the > memory use but that doesn't make a dent in 4GB of allocations. Right. I didn't expect much from that fix. Nicolas - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
