On 11/20/2017 6:51 PM, Ramsay Jones wrote:
On 20/11/17 14:01, Ben Peart wrote:
Further testing has revealed that switching from the regular heap to a refactored version of the mem_pool in fast-import.c produces similar gains as parallelizing do_index_load(). This appears to be a much simpler patch for similar gains so we will be pursuing that path.
Combining the two patches resulted in a further 25% reduction of do_index_load() but with index load times getting smaller, that only resulted in a 3% improvement in total git command time. Given the greater complexity of this patch, we've decided to put it on hold for now. We'll keep it in our back pocket in case we need it in the future.
Since you are withdrawing the 'bp/fastindex' branch, I won't send the
patch to fix up the sparse warnings (unless you would like to see it
before you put it in your back pocket). ;-)
ATB,
Ramsay Jones
I'm happy to take your patches as I am keeping a copy just in case we
need to revisit it.
I'm actually interested to see how Peff's suggestion to use tcmalloc
helps because nearly 90% of the remaining execution time was spent
waiting for a lock in malloc due to all contention from the multi-threading.
