On Mon, Jul 11, 2022 at 05:15:37PM +0200, Olaf Hering wrote: > I ran top(1), all the git processes were competing for memory. > There was most of the time no memory shared, according to top. OK, that makes sense if they really are allocating a lot of heap. They _would_ be sharing mmap'd pages, but the memory pressure causes the OS to evict as many of those as possible. > This tool to process a single repository exists since a few years. > In the past I ran it on a 12cpu/64GB machine. Today it was running the > first time on a 96cpu/64GB machine. There is a slim chance the issue did > not show up because the 12 processes always had access to enough memory. > > I will try your suggestions. In case you haven't seen it, read the end of: https://lore.kernel.org/git/YsxiSwQGvLhzNQrt@xxxxxxxxxxxxxxxxxxxxxxx/ I realized my reproduction on linux.git was not traversing a wide enough chunk of history. Fixing that, I do see ~1GB of heap allocation. My patch drops that substantially, but you will still be much better off building a commit-graph file. -Peff
