On Tue, 3 Sep 2013, Duy Nguyen wrote: > On Tue, Sep 3, 2013 at 6:49 PM, Duy Nguyen <pclouds@xxxxxxxxx> wrote: > > On Tue, Sep 3, 2013 at 1:46 PM, Nicolas Pitre <nico@xxxxxxxxxxx> wrote: > >> So... looks like pack v4 is now "functional". > >> > >> However something is still wrong as it operates about 6 times slower > >> than pack v3. > >> > >> Anyone wishes to investigate? > > > > You recurse in decode_entries too deep.I check the first 10000 > > decode_entries() calls in pv4_get_tree(). The deepest level is 3491. > > And I was wrong, the call depth is not that deep, but the number of > decode_entries calls triggered by one pv4_get_tree() is that many. > This is on git.git and the tree being processed is "t", which has 672 > entries.. There are funny access patterns. This is the output of > > fprintf(stderr, "[%d] %d - %d %u\n", call_depth, copy_start, > copy_count, copy_objoffset); > > [1] 0 - 1 48838573 > [2] 0 - 1 48826699 > [3] 0 - 1 48820760 > [4] 0 - 1 48814812 > [5] 0 - 1 48805904 > [6] 0 - 1 48797000 > [7] 0 - 1 48794034 > [8] 0 - 1 48791067 > [9] 0 - 1 48788100 > [10] 0 - 1 48785134 > [11] 0 - 1 48776221 > [12] 0 - 1 48764321 > [13] 0 - 1 48503227 > [14] 0 - 1 48485415 > [15] 0 - 1 48473512 > [16] 0 - 1 48443621 > [17] 0 - 1 48401788 > [18] 0 - 1 48377834 > [19] 0 - 1 48371841 > [20] 0 - 1 48341809 > [21] 0 - 1 48260734 > [22] 0 - 1 48236635 > [23] 0 - 1 46845105 > [24] 0 - 1 14603061 > [25] 2 - 1 48838573 > [2] 0 - 1 48826699 > > It goes through 20+ base trees just to get one tree entry, I think.. Yeah... that's true. The encoding should refer to the deepest tree directly in that case. Better delta heuristics will have to be worked out here. The code as it is now can't do that. There was also a bug that prevented larger copy sequences to be created which is now fixed. I added to packv4-create the ability to specify the minimum range of consecutive entries that can be represented by a copy sequence to allow experiments. However, even when the tree deltas are completely disabled (using --min-tree-copy=0 achieves that) the CPU usage is still much higher which is rather unexpected. In theory this shouldn't be the case. Many other bugs have now been fixed. A git.git repository with packs version 4 appears to be functional and passes git-fsck --full --strict. Nicolas -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html