On 12/7/07, Nicolas Pitre <nico@xxxxxxx> wrote: > On Fri, 7 Dec 2007, david@xxxxxxx wrote: > > > On Fri, 7 Dec 2007, Jon Smirl wrote: > > > > > I noticed two things when doing a repack of the gcc repo. First is > > > that the git process is getting to be way too big. Turning off the > > > delta caches had minimal impact. Why does the process still grow to > > > 4.8GB? > > > > > > Putting this in perspective, this is a 4.8GB process constructing a > > > 330MB file. Something isn't right. Memory leak or inefficient data > > > structure? > > > > keep in mind that that 330MB file is _very_ heavily compressed. the simple > > zlib compression is probably getting you 10:1 or 20:1 compression and the > > delta compression is a significant multiplier on top of that. > > Doesn't matter. Something is indeed fishy. I didn't have any problem repacking Mozilla and it ends up as a 450MB pack file with 1.5M entries. So something has changed. With Mozilla I had a 3GB machine, and now I can't finish a 330MB pack on a 4GB machine. I don't recall the Mozilla process ever exceeding 2GB. > > The bulk of pack-objects memory consumption can be estimated as follows: > > 1M objects * sizeof(struct object_entry) ~= 100MB > 256 window entries with data (assuming a big 1MB per entry) = 256MB > Delta result caching was disabled therefore 0MB > read-side delta cache limited to 16MB > > So the purely ram allocation might get to roughly 400MB. > > Then add the pack and index map, which, depending on the original pack > size, > might be 2GB. > > So we're pessimistically talking of about 2.5GB of virtual space. > > The other 2.3GB is hard to explain. > > > Nicolas > -- Jon Smirl jonsmirl@xxxxxxxxx - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
