Re: pack operation is thrashing my server

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On Wed, 13 Aug 2008, Nicolas Pitre wrote:
> 
> A much better solution would consist of finding just _why_ object 
> enumeration is so slow.  This is indeed my biggest grip with git 
> performance at the moment.
> 
> |nico@xanadu:linux-2.6> time git rev-list --objects --all > /dev/null
> |
> |real    0m21.742s
> |user    0m21.379s
> |sys     0m0.360s
> 
> That's way too long for 1030198 objects (roughly 48k objects/sec).

Why do you think that's horribly slow?

Doing a rev-list of all objects is a fairly rare operation, but even if 
you want to clone/repack all of your archives the whole time, please 
realize that listing objects is _not_ a simple operation. It opens up and 
parses every single tree in the whole history. That's a _lot_ of data to 
unpack.

And trees also pack very efficiently (because they delta so well), so 
there's a lot of complex ops there.

> And it gets even worse with the gcc repository:

I bet it's because gcc has a different directory structure. I don't have 
the gcc sources in front of me, but I'd suspect something like a single 
large directory or other.

> Clearly something is not scaling here.

I don't agree. There's no "clearly" about it. Different data sets.

		Linus
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux