Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx> writes: > On Tue, 11 Dec 2007, Jon Smirl wrote: >> > >> > So if you want to use more threads, that _forces_ you to have a bigger >> > memory footprint, simply because you have more "live" objects that you >> > work on. Normally, that isn't much of a problem, since most source files >> > are small, but if you have a few deep delta chains on big files, both the >> > delta chain itself is going to use memory (you may have limited the size >> > of the cache, but it's still needed for the actual delta generation, so >> > it's not like the memory usage went away). >> >> This makes sense. Those runs that blew up to 4.5GB were a combination >> of this effect and fragmentation in the gcc allocator. Google >> allocator appears to be much better at controlling fragmentation. > > Yes. I think we do have some case where we simply keep a lot of objects > around, and if we are talking reasonably large deltas, we'll have the > whole delta-chain in memory just to unpack one single object. Eh, excuse me. unpack_delta_entry() - first unpacks the base object (this goes recursive); - uncompresses the delta; - applies the delta to the base to obtain the target object; - frees delta; - frees (but allows it to be cached) the base object; - returns the result So no matter how deep a chain is, you keep only one delta at a time in core, not whole delta-chain in core. > So what you actually want to do is to just re-use already packed delta > chains directly, which is what we normally do. But you are explicitly > looking at the "--no-reuse-delta" (aka "git repack -f") case, which is why > it then blows up. While that does not explain, as Nico pointed out, the huge difference between the two repack runs that have different starting pack, I would say it is a fair thing to say. If you have a suboptimal pack (i.e. not enough reusable deltas, as in the 2.1GB pack case), do run "repack -f", but if you have a good pack (i.e. 300MB pack), don't. - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
