Re: Using bitmaps to accelerate fetch and clone

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Sep 27, 2012 at 7:47 AM, Shawn Pearce <spearce@xxxxxxxxxxx> wrote:
> * https://git.eclipse.org/r/7939
>
>   Defines the new E003 index format and the bit set
>   implementation logic.

Quote from the patch's message:

"Currently, the new index format can only be used with pack files that
contain a complete closure of the object graph e.g. the result of a
garbage collection."

You mentioned this before in your idea mail a while back. I wonder if
it's worth storing bitmaps for all packs, not just the self contained
ones. We could have one leaf bitmap per pack to mark all leaves where
we'll need to traverse outside the pack. Commit leaves are the best as
we can potentially reuse commit bitmaps from other packs. Tree leaves
will be followed in the normal/slow way.

For connectivity check, fewer trees/commits to deflate/parse means
less time. And connectivity check is done on every git-fetch (I
suspect the other end of a push also has the same check). It's not
unusual for me to fetch some repos once every few months so these
incomplete packs could be quite big and it'll take some time for gc
--auto to kick in (of course we could adjust gc --auto to start based
on the number of non-bitmapped objects, in additional to number of
packs).
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]