Re: Efficiency of initial clone from server

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 2/12/07, Shawn O. Pearce <spearce@xxxxxxxxxxx> wrote:
Jon Smirl <jonsmirl@xxxxxxxxx> wrote:
> But pack to the original point, can't the server check and see if it
> has write access so that it can keep the fully packed tree? I've just
> caused kernel.org to needlessly repack the wireless-dev tree a dozen
> times playing with this clone command. If it didn't have to keep
> repacking for the clone, clone would be a lot faster.

We probably could.

I have actually been thinking about another problem that is
somewhat related.  We cannot put more than 4 GiB of data into a
single packfile, due to the current index size limitation, or more
than 2^32-1 objects into one packfile, due to the header nr_objects
field size.

Right now we are sending a single packfile down to the client,
even if the remote server end has the repository broken down into
a couple of packfiles (such as "really old historical stuff" and
"active stuff from this year").  If we could send more than one
packfile to the client in a single stream, we could still keep the
file size limitations.

We can also avoid this huge repack case on the server.  Because it
could just send all of the packfiles that it already has, followed
by whatever is loose which wasn't in a prior packfile.  And no
write access required.

Of course, we still could do the optimization of caching the
packfile, but I'm not sure how well that would work on kernel.org,
as I understand the trees are owned by the devs which created them
while the git daemon is probably not running as their UNIX user.

I didn't want to cache the packfile, instead I wanted to repack the
repository and then copy the resulting pack file down the wire. A
clone would just be a trigger to make sure everything in the repo was
packed (maybe into multiple packs) before starting to send anything.
Doing it this way means that everyone benefits from the packing.



--
Shawn.



--
Jon Smirl
jonsmirl@xxxxxxxxx
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]