On Thu, 05 Sep 2024, Jani Nikula <jani.nikula@xxxxxxxxxxxxxxx> wrote: > On Thu, 05 Sep 2024, Jonathan Corbet <corbet@xxxxxxx> wrote: >> Jani Nikula <jani.nikula@xxxxxxxxxxxxxxx> writes: >> >>>> However my patch for sphinx -j1 did give good test results, too. >>>> Maybe in your case that would result in 8 sphinx calls with 1 thread >>>> each, which would be more appropriate for your machine. >>> >>> The right thing to do is to have one sphinx-build process and pass -j<N> >>> to that. >> >> [I wouldn't have minded being CC'd on this conversation...] > > Yes, sorry about that. > >> I, too, have never seen the behavior Jani reports. >> >> I, too, would like to get rid of as much of the makefile hackery as >> possible, but it all did end up there for a reason. >> >> The business around parallelism was intended to make sphinx play well >> with other targets being built in the same make invocation. If you do a >> "make -j8 this that theother htmldocs" and there are five processes >> working on this, that, and theother, then sphinx should not create more >> than three. See 51e46c7a4007 for more. >> >> In Jani's case, it sounds like the job-slot reservation isn't working >> right somehow? > > Looking at the -j<N> results on an unrelated project, PEBKAC is a > distinct possibility here, and this part may be a red herring. I'll need > to look into it. And just so there's no confusion, this is purely about my comments. The problem with reproducible builds Bernhard presents is still a problem. One of the key questions is whether we end up launching multiple sphinx-build processes ourselves (but maybe some other way than I speculated), making this a duplicate of [1], or whether plain single sphinx-build -j<N> on its own *also* has parallel build issues. And if the latter, can they be caused by our extensions incorrectly indicating parallel_read_safe = True or parallel_write_safe = True in their setup() functions, or is it something inherently in Sphinx. BR, Jani. [1] https://github.com/sphinx-doc/sphinx/issues/2946 -- Jani Nikula, Intel
