Re: Surprising use of memory and time when repacking mozilla's gecko repository

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Jul 04, 2019 at 07:05:30PM +0900, Mike Hommey wrote:
> My guess is all those stalls are happening when processing the files I
> already had problems with in the past[3], except there are more of them
> now (thankfully, they were removed, so there won't be more, but that
> doesn't make the existing ones go away).
>
> 3. https://public-inbox.org/git/20180703223823.qedmoy2imp4dcvkp@xxxxxxxxxxxx/T/

I've more or less confirmed that's the cause of the long stalls during
the compression phase. It can be reproduced to some extent with:

$ git clone https://github.com/mozilla/gecko
$ cd gecko
$ git rev-list --all -- testing/web-platform/meta/MANIFEST.json | xargs -I '{}' git ls-tree '{}' testing/web-platform/meta/MANIFEST.json | sort -u | awk '{print $3}' | git cat-file --batch-check | sort -n -k 3 | awk '{print $1}' | git pack-objects --window=250 --depth=50 --no-reuse-delta my_pack

There might be some other file than testing/web-platform/meta/MANIFEST.json 
involved, because this uses "only" 40GB RAM, but that file is the one I know
of.

This however doesn't reproduce the "writing takes forever" part of the
problem.

Mike



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux