Em Thu, 19 Sep 2019 11:18:52 -0700
Kees Cook <keescook@xxxxxxxxxxxx> escreveu:
> While sphinx 1.7 and later supports "-jauto" for parallelism, this
> effectively ignores the "-j" flag used in the "make" invocation, which
> may cause confusion for build systems. Instead, extract the available
> parallelism from "make"'s job server (since it is not exposed in any
> special variables) and use that for the "sphinx-build" run. Now things
> work correctly for builds where -j is specified at the top-level:
>
> make -j16 htmldocs
>
> Signed-off-by: Kees Cook <keescook@xxxxxxxxxxxx>
> ---
> Documentation/Makefile | 3 +--
> scripts/jobserver-count | 46 +++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 47 insertions(+), 2 deletions(-)
> create mode 100755 scripts/jobserver-count
>
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index e145e4db508b..4408eeaf2891 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -33,8 +33,6 @@ ifeq ($(HAVE_SPHINX),0)
>
> else # HAVE_SPHINX
>
> -export SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
> -
> # User-friendly check for pdflatex and latexmk
> HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi)
> HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi)
> @@ -68,6 +66,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
> PYTHONDONTWRITEBYTECODE=1 \
> BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
> $(SPHINXBUILD) \
> + -j $(shell $(srctree)/scripts/jobserver-count) \
> -b $2 \
> -c $(abspath $(srctree)/$(src)) \
> -d $(abspath $(BUILDDIR)/.doctrees/$3) \
> diff --git a/scripts/jobserver-count b/scripts/jobserver-count
> new file mode 100755
> index 000000000000..5fc9d2fd5254
> --- /dev/null
> +++ b/scripts/jobserver-count
> @@ -0,0 +1,46 @@
> +#!/usr/bin/env python3
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# This determines how many parallel tasks "make" is expecting, as it is
> +# not exposed via an special variables.
> +# https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
> +import os, sys, fcntl
> +
> +# Set non-blocking for a given file descriptor.
> +def nonblock(fd):
> + flags = fcntl.fcntl(fd, fcntl.F_GETFL)
> + fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
> + return fd
> +
> +# Fetch the make environment options.
> +flags = os.environ.get('MAKEFLAGS', None)
> +if flags == None:
> + print("1")
> + sys.exit(0)
> +
> +# Look for "--jobserver=R,W"
> +opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
> +if len(opts) != 1:
> + print("1")
> + sys.exit(0)
Using "1" as default if -j is not specified doesn't sound a good idea. I
mean, Sphinx is very slow without any parallelism. I would keep "auto" as
default here, if version >= 1.7 (and if there's no explicit SPHINXOPTS).
> +
> +# Parse out R,W file descriptor numbers and set them nonblocking.
> +fds = opts[0].split("=", 1)[1]
> +reader, writer = [nonblock(int(x)) for x in fds.split(",", 1)]
> +
> +# Read out as many jobserver slots as possible.
> +jobs = b""
> +while True:
> + try:
> + slot = os.read(reader, 1)
> + jobs += slot
> + except:
> + break
> +# Return all the reserved slots.
> +os.write(writer, jobs)
> +
> +# Report available slots (with a bump for our caller's reserveration).
> +if len(jobs) < 1:
> + print("1")
> +else:
> + print(len(jobs) + 1)
Thanks,
Mauro
