On 10/29/19 1:16 AM, Markus Heiser wrote: > Am 28.10.19 um 20:07 schrieb Kees Cook: >> On Sun, Oct 27, 2019 at 04:05:07PM -0300, Mauro Carvalho Chehab wrote: >>> Em Sun, 27 Oct 2019 11:30:22 -0700 >>> Randy Dunlap <rdunlap@xxxxxxxxxxxxx> escreveu: >>> >>>> On 10/27/19 11:17 AM, Jonathan Corbet wrote: >>>>> On Sun, 27 Oct 2019 11:11:57 -0700 >>>>> Randy Dunlap <rdunlap@xxxxxxxxxxxxx> wrote: >>>>> >>>>>>> Yes, I see oom-kills when building docs for 5.4-rc2. >>>>>>> But only after I ran 'gimp *.jpg' for 22 photos. >>>>>>> >>>>>>> I can see anywhere from 1 to 4 sphinx-build processes running. >>>>>> >>>>>> I continue to have oom-kills when building docs. >>> >>> My 2 cents here. >>> >>> I would keep the default to use more CPUs, as nowadays I suspect that >>> most devs have a way more than 8GB installed. The worse machine I use >>> here to build Kernel is a Gen-3 i5 that has has 16GB (and it is a >>> personal laptop I bought 5 years ago). >>> >>> Yet, I agree that doc build should be saner on low-cap machines. >>> > > In the past I run sphinx builds on a small 4GB VM (2cpu) without any problems. > (nowadays) 16 CPU, 16 GB PC with sphinx-build the max load of my whole PC is > about 5GB (*more CPUs need more mem*). > > @Randy you said the build machine hase 8GB .. and how much CPUs? 4. >>> Ideally, the fix should be, IMHO, at sphinx-build: -jauto should >>> detect OOM possible conditions and reduce the number of used CPUs >>> it the max available memory is too low. >> >> How can we determine that dynamically, though? e.g. how do we determine >> how much is needed per thread in a way that will move with the future >> needs of Sphinx, etc? > > d'accord > >>> As there's now a script at Kernel, maybe the script could be modified >>> to detect if the max available memory is too low, passing -j1 if, >>> let's say, the machine has less than (let's say) 16 GB RAM. >> >> Randy, are you able to just use -j2 or similar with your builds to avoid >> this? No, sometimes even -j1 fails with oom-killer. > > Sphinx-doc is pure Python, it runs a lot of regexpr and is moving string, > nothing special .. maybe we have a mem leak in one of our extensions? > > @Kess: why did you mentioning MAINTAINERS parsing .. did you observe anything > strange? > > Maybe one of the tools used by sphinx is buggy (latex, dot, image-magic) .. needs investigation. > > @Randy: my first questions are: > > - cpu / mem? > - are OOM-kills when building target htmldocs, pdfdocs or ..? htmldocs > - which py & sphinx version are used? Reference installation see 'virtualenv' > in [1] Python 2.7.14 sphinx-build 1.7.6 > -- Markus -- > > > [1] https://www.kernel.org/doc/html/latest/doc-guide/sphinx.html#sphinx-install thanks. -- ~Randy
