On Mon, 9 Jan 2023 15:14:46 +0100, Martin Liška wrote: > On 1/8/23 15:01, Mauro Carvalho Chehab wrote: >> Em Sat, 7 Jan 2023 14:17:24 +0900 >> Akira Yokosawa <akiyks@xxxxxxxxx> escreveu: >> >>> On Wed, 04 Jan 2023 13:45:35 -0700, Jonathan Corbet wrote: >>>> Sphinx 6.0 removed the execfile_() function, which we use as part of the >>>> configuration process. They *did* warn us... Just open-code the >>>> functionality as is done in Sphinx itself. >>>> >>>> Tested (using SPHINX_CONF, since this code is only executed with an >>>> alternative config file) on various Sphinx versions from 2.5 through 6.0. >>>> >>>> Reported-by: Martin Liška <mliska@xxxxxxx> >>>> Signed-off-by: Jonathan Corbet <corbet@xxxxxxx> >>> >>> I have tested full builds of documentation with this change >>> with Sphinx versions 1.7.9, 2.4.5, 3.4.3, 4.5.0, 5.3.0, and 6.0.0. >>> >>> Tested-by: Akira Yokosawa <akiyks@xxxxxxxxx> >>> >>> That said, Sphinx 6.0.0 needs much more time and memory than earlier >>> versions. >>> >>> FYI, I needed to limit parallel slot to 2 (make -j2) on a 16GB machine. >>> If you are lucky, -j3 and -j4 might succeed. -j5 or more ended up in >>> OOM situations for me: >>> >>> Comparison of elapsed time and maxresident with -j2: >>> >>> ============== ============ =========== >>> Sphinx version elapsed time maxresident >>> ============== ============ =========== >>> 5.3.0 10:16.81 937660 >>> 6.0.0 17:29.07 5292392 >>> ============== ============ =========== > > Hi. > > I can confirm the regression, I bisected Sphinx revision that caused that > and filled an upstream issues: > https://github.com/sphinx-doc/sphinx/issues/11116 Thank you Martin for looking into this! > > Cheers, > Martin > >> >> From the changelogs: >> https://www.sphinx-doc.org/en/master/changes.html >> >> It seems that 6.1 came with some performance optimizations, in particular: >> >> Cache doctrees in the build environment during the writing phase. >> >> Make all writing phase tasks support parallel execution. >> >> Cache doctrees between the reading and writing phases. >> >> It would be nice if you could also test and check elapsed time >> there too, as I suspect that 6.0 will have a very short usage, as >> 6.1 was released just a few days after it. Here is a table comparing 5.3.0, 6.0.1 and 6.1.2 taken on the same machine (with 16GiB mem + 2GiB swap): ====== =================================== elapsed time ----------------------------------- Sphinx -j1 -j2 -j4 -j6 ====== ======== ======== ======== ======== 6.1.2 15:11.74 18:06.89 16:39.93 OOM 6.0.1 15:28.19 17:22.15 16:31.30 OOM 5.3.0 14:13.04 10:16.81 8:22.37 8:09.74 ====== ======== ======== ======== ======== Note: - The -j1 run needs an explicit option given to sphinx-build by: make SPHINXOPTS="-q -j1" htmldocs - Once an OOM happens, -j4 runs are likely end up in OOM. Looks like the non-parallel run is the cheapest option for mitigating the regression. Thanks, Akira >> >> Regards, >> Mauro. >> >> >> >>> >>> Thanks, Akira
