On Thu, 05 Sep 2024, Vegard Nossum <vegard.nossum@xxxxxxxxxx> wrote: > On 05/09/2024 14:20, Jani Nikula wrote: >> On Thu, 05 Sep 2024, Jani Nikula <jani.nikula@xxxxxxxxxxxxxxx> wrote: >>> On Thu, 05 Sep 2024, bernhard+linux-doc@xxxxxxxx wrote: >>>> From: "Bernhard M. Wiedemann" <bwiedemann@xxxxxxx> >>>> >>>> Because we want reproducible builds >>>> and https://github.com/sphinx-doc/sphinx/issues/6714 >>>> did not receive any love from Sphinx devs in five years, >>>> let's disable parallel doc builds until that Sphinx issue is fixed. >>> >>> You mention in [1] that this is likely a duplicate of [2] i.e. multiple >>> Sphinx instances running in parallel and racing in doctree access. >>> >>> In kernel, does the issue then boil down to: >>> >>> htmldocs: >>> @$(srctree)/scripts/sphinx-pre-install --version-check >>> @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var))) >>> >>> i.e. multiple Sphinx invocations instead of just one? >>> >>> Broken record, I still think we should axe the Makefile hacks out of the >>> Sphinx build altogether, and let Sphinx handle everything, via >>> extensions if necessary. >> >> Indeed. On my machine, 'make htmldocs' spawns eight sphinx-build >> processes, each running with -j8, and each racing for the same >> doctrees. And the whole thing is apparently parallelized to 64 threads, >> which is unlikely to be the most efficient. >> >> There's no reason to blame Sphinx upstream before we get our end >> together, and run Sphinx as it was designed. > > I don't reproduce this here, either with plain 'make htmldocs' or 'make > -j8 htmldocs' -- in either case the process tree is: > > make ... htmldocs > - make ... htmldocs > - make -f ./scripts/Makefile.build obj=Documentation htmldocs > - /bin/sh -c make ... > - python3 ./scripts/jobserver-exec sh > ./Documentation/sphinx/parallel-wrapper.sh sphinx-build -b html ... > - /usr/bin/python3 /usr/bin/sphinx-build -j... -b html ... > > Neither the $(foreach) in the Makefile nor $(call loop_cmd) should > result in parallel invocations. > > Could you double check the process tree once the "Using alabaster theme" > message has appeared? $ pstree -A 3457144 bash---make---make---make---make---sh---python3---sphinx-build-+-7*[sphinx-build] `-sphinx-build---perl Maybe try after 'make cleandocs'. BR, Jani. > > > Vegard -- Jani Nikula, Intel
