Re: pack operation is thrashing my server

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On Thu, 14 Aug 2008, Andi Kleen wrote:
> 
> I would have thought that zlib has a sequential access pattern that the
> CPU prefetchers have a easy time with hiding latency.

No, the lookup tables for the patterns are quite non-sequential. It does 
do a lot of indirect accesses, ie it loads data from the input stream and 
then looks things up through that. 

But it's quite possible that we should use different compression factors 
for different object types. Right now we have different (configurable) 
compression levels for loose objects and packs, but it might be 
interesting to see what happens for just "packed tree objects".

The trees really end up having rather different access patterns in 
pack-files. They also tend to be rather less compressible than other 
blobs, since the SHA1's in there are just random binary data. They also 
delta very well - obviously regular blobs do that _too_, but regular blobs 
are seldom as performance-critical in git (ie once you actually unpack a 
blob, there are other things going on like actually generating a diff - 
but trees get unpacked over and over for "internal git reasons")

		Linus
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux