On Wed, Sep 26, 2018 at 9:54 PM Ben Peart <peartben@xxxxxxxxx> wrote: > The high cost of parsing the index extensions is driven by the cache tree > and the untracked cache extensions. As this is currently the longest pole, > any reduction in this time will reduce the overall index load times so is > worth further investigation in another patch series. > > Name First Last Elapsed > | + git!read_index_extension 684.052 870.244 186.192 > | + git!cache_tree_read 684.052 797.801 113.749 > | + git!read_untracked_extension 797.801 870.244 72.443 > > One option would be to load each extension on a separate thread but I > believe that is overkill for the vast majority of repos. They both grow proportional to the number of trees in worktree, which probably also scales to the worktree size. Frankly I think the parallel index loading is already overkill for the majority of repos, so speeding up further of the 1% giant repos does not sound that bad. And I think you already lay the foundation for loading index stuff in parallel with this series. > Instead, some > optimization of the loading code for these two extensions is probably worth > looking into as a quick examination shows that the bulk of the time for both > of them is spent in xcalloc(). Another easy "optimization" is delaying loading these until we need them (or load them in background, read_index() returns even before these extensions are finished, but this is of course trickier). UNTR extension for example is only useful for "git status" (and maybe one or two other use cases). Not having to load them all the time is likely a win. The role of TREE extension has grown bigger these days so it's still maybe worth putting more effort into making it load it faster rather than just hiding the cost. -- Duy
