Am 28.10.19 um 20:07 schrieb Kees Cook:
On Sun, Oct 27, 2019 at 04:05:07PM -0300, Mauro Carvalho Chehab wrote:
Em Sun, 27 Oct 2019 11:30:22 -0700
Randy Dunlap <rdunlap@xxxxxxxxxxxxx> escreveu:
On 10/27/19 11:17 AM, Jonathan Corbet wrote:
On Sun, 27 Oct 2019 11:11:57 -0700
Randy Dunlap <rdunlap@xxxxxxxxxxxxx> wrote:
Yes, I see oom-kills when building docs for 5.4-rc2.
But only after I ran 'gimp *.jpg' for 22 photos.
I can see anywhere from 1 to 4 sphinx-build processes running.
I continue to have oom-kills when building docs.
My 2 cents here.
I would keep the default to use more CPUs, as nowadays I suspect that
most devs have a way more than 8GB installed. The worse machine I use
here to build Kernel is a Gen-3 i5 that has has 16GB (and it is a
personal laptop I bought 5 years ago).
Yet, I agree that doc build should be saner on low-cap machines.
In the past I run sphinx builds on a small 4GB VM (2cpu) without any problems.
(nowadays) 16 CPU, 16 GB PC with sphinx-build the max load of my whole PC is
about 5GB (*more CPUs need more mem*).
@Randy you said the build machine hase 8GB .. and how much CPUs?
Ideally, the fix should be, IMHO, at sphinx-build: -jauto should
detect OOM possible conditions and reduce the number of used CPUs
it the max available memory is too low.
How can we determine that dynamically, though? e.g. how do we determine
how much is needed per thread in a way that will move with the future
needs of Sphinx, etc?
d'accord
As there's now a script at Kernel, maybe the script could be modified
to detect if the max available memory is too low, passing -j1 if,
let's say, the machine has less than (let's say) 16 GB RAM.
Randy, are you able to just use -j2 or similar with your builds to avoid
this?
Sphinx-doc is pure Python, it runs a lot of regexpr and is moving string,
nothing special .. maybe we have a mem leak in one of our extensions?
@Kess: why did you mentioning MAINTAINERS parsing .. did you observe anything
strange?
Maybe one of the tools used by sphinx is buggy (latex, dot, image-magic) ..
needs investigation.
@Randy: my first questions are:
- cpu / mem?
- are OOM-kills when building target htmldocs, pdfdocs or ..?
- which py & sphinx version are used? Reference installation see 'virtualenv'
in [1]
-- Markus --
[1] https://www.kernel.org/doc/html/latest/doc-guide/sphinx.html#sphinx-install
