On Tue, Feb 18, 2014 at 3:55 PM, David Kastrup <dak@xxxxxxx> wrote: > Christian Jaeger <chrjae@xxxxxxxxx> writes: > >> I've got a repository where "git log --raw > _somefile" took a few >> seconds in the past, but after an attempt at merging some commits that >> were collected in a clone of the same repo that was created about a >> year ago, I noticed that this command was now taking 3 minutes 7 >> seconds. "git gc", "git fsck", "git clone file:///the/repo/.git" also >> now each took between ~4-10 minutes, also "git log --raw somefile" got >> equally unusably slow. With the help of the people on the IRC, I >> tracked it down to my recent use of "git gc --aggressive" in this >> repo. Running "git repack -a -d -f" solved it, now it's again taking >> 4-5 seconds. After running "git gc --aggressive" again for >> confirmation, "git log --raw > _somefile" was again slowed down, >> although now 'only' to 1 minute 34 seconds; > > [...] > >> I've now learned to avoid "git gc --aggressive". Perhaps there are >> some other conclusions to be drawn, I don't know. > > I've seen the same with my ongoing work on git-blame with the current > Emacs Git mirror. Aggressive packing reduces the repository size to > about a quarter, but it blows up the system time (mainly I/O) > significantly, quite reducing the total benefits of my algorithmic > improvements there. Likely because --aggressive passes --depth=250 to pack-objects. Long delta chains could reduce pack size and increase I/O as well as zlib processing signficantly. Christian can try "git repack -adf" which is really close to --aggressive (except it uses default --depth=50) and see if it makes any difference. > There is also some quite visible additional time spent in zlib, so a > wild guess would be that zlib is not really suited to the massive amount > of directory entries of a Git object store. Since the system time still > dominates, this guess would only make sense if Git over zlib kept > rereading the directory section of whatever compressed file we are > talking about. But that's really a rather handwavy wild guess without > anything better than a hunch to back it up. I don't even know what kind > of compression and/or packs are used: I've only ever messed myself with > the delta coding of the normal "unpacked" operation (there are a few > older commits from me on that). > > -- > David Kastrup > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html