Re: How to unpack recent objects?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, 16 Dec 2010, Phillip Susi wrote:

> On 12/16/2010 4:19 PM, Nicolas Pitre wrote:
> > What makes you think that unpacking them will actually make the access 
> > to them faster?  Instead, you should consider _repacking_ them, 
> > ultimately using the --aggressive parameter with the gc command, if you 
> > want faster accesses.
> 
> Because decompressing and undeltifying the objects in the pack file
> takes a fair amount of cpu time.  It seems a waste to do this for the
> same set of objects repeatedly rather than just keeping them loose.

Well, here are a couple implementation details you might not know about:

1) Loose objects are compressed too.  So you gain nothing on that front 
   by keeping objects loose.

2) Delta ordering is so that recent objects, i.e. those belonging to 
   most recent commits, are not delta compressed but rather used as base 
   objects for "older" objects to delta against.  So in practice, the 
   cost of undeltifying objects is pushed towards objects that you're 
   most unlikely to access frequently.

3) Object placement within the pack is also optimized so that 
   objects belonging to recent commits are close together, and walking 
   them creates a linear IO access pattern which is much faster than 
   accessing random individual files as loose objects are.

4) Packed objects take considerably less space than loose ones which 
   makes for much better usage of the file system cache in the operating 
   system.  This largely outweights the cost of undeltifying objects.

5) Git also keeps a cache of most frequently referenced objects when 
   replaying delta chains so deep deltas don't bring exponential costs.

And, in some cases, Git does even pick up the content of an object by 
using its checked out form in the working directory directly instead of 
locating and decompressing the object data.

So you shouldn't have to worry on that front.  Git is not the fastest 
SCM out there just by luck.


Nicolas
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]