Re: [RFC] super indexes to span multiple packfiles

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Object's are not accessed in random order with git. Once an object
reference hits a pack file it is very likely that following references
will hit the same pack file. That's because you always find object
SHA's by following the chains.

So first place to look for an object is the same place the previous
object was found. If it isn't there order the search of the pack files
by creation data (just a heuristic). Make this list a circle and start
the search in the pack where the previous object was found. This can
all be done with the existing indexes.

I haven't been reading all of the messages on this subject, but is
this strategy enough to eliminate the need for a super index?

If you still need a super index, note that it may be good enough for
it to only contain the SHA's for objects that are externally
referenced. This index would be small and simply point to the correct
pack file index to find the object in. You could add a list of
dangling links to each packfile index to assist with building this
super index.

My work with databases leads me to believe that figuring out how to
pack everything into a smaller space always beats efforts put into
incrementally improving the indexing scheme. Packing into a smaller
space reduces the total IO needs and that's always a winner.

--
Jon Smirl
jonsmirl@xxxxxxxxx
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux