Re: pack operation is thrashing my server

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Andreas Ericsson wrote:
> Nicolas Pitre wrote:
> > |nico@xanadu:linux-2.6> time git rev-list --objects --all > /dev/null
> > |
> > |real    0m21.742s
> > |user    0m21.379s
> > |sys     0m0.360s
> > 
> > That's way too long for 1030198 objects (roughly 48k objects/sec).  And 
> > it gets even worse with the gcc repository:
> > 
> > |nico@xanadu:gcc> time git rev-list --objects --all > /dev/null
> > |
> > |real    1m51.591s
> > |user    1m50.757s
> > |sys     0m0.810s
> > 
> > That's for 1267993 objects, or about 11400 objects/sec.
> > 
> > Clearly something is not scaling here.
> > 
> 
> What are the different packing options for the two repositories?
> A longer deltachain and larger packwindow would increase the
> enumeration time, wouldn't it?

For the fun of it, I ran a test without deltas.  Here's my normal
git.git:

  $ du -h .git/objects/pack
  26M     .git/objects/pack
  $ git rev-list --all | wc -l
  17638
  $ git rev-list --all --objects | wc -l
  82194

On a hot cache I get about 61800 objects/sec:

  $ /usr/bin/time git rev-list --all --objects >/dev/null
  1.33user 0.04system 0:01.39elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
  0inputs+0outputs (0major+8087minor)pagefaults 0swaps

I then made a copy of that and repacked it without deltas (remember to
remove *.keep, I tripped over that twice):

  $ git repack --depth=0 --window=0 -a -f -d
  Counting objects: 82906, done.
  Writing objects: 100% (82906/82906), done.
  Total 82906 (delta 0), reused 0 (delta 0)
  $ du -h .git/objects/pack
  339M    .git/objects/pack

Which results in only 28739 objects/sec:

  $ /usr/bin/time git rev-list --all --objects >/dev/null
  2.86user 0.11system 0:02.98elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
  0inputs+0outputs (0major+50162minor)pagefaults 0swaps

So maybe the GCC repository would need to be packed _better_?

Unfortunately I cannot sensibly run the same test on linux-2.6.git,
which is the next bigger git I have around: it inflates to about 3GB
after the repack, which does not fit into memory.

- Thomas

-- 
Thomas Rast
trast@xxxxxxxxxxxxxxx

Attachment: signature.asc
Description: This is a digitally signed message part.


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux