parallelization of ./configure compiler test processes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello autoconf,

I work on a cross-platform package manager named spack (https://spack.io) which builds lots of gnu software from source and has fantastic support for autotools projects. Because spack provides a shell script `cc` to wrap the compiler, each invocation of `cc` for feature tests executed by `./configure` takes a little bit longer than normal, so configuring projects that necessarily have a lot of feature tests takes much longer in spack (particularly `gettext`, which we use as a benchmark in this change: https://github.com/spack/spack/pull/26259). However, we can fix that additional overhead ourselves without any changes in autoconf, by generating our `cc` wrapper instead of doing any logic in the shell script. The reason I messaged this board is because of a separate idea that the above situation made me start thinking about: *parallelizing feature test executions*, in order to speed up `./configure`.

So a few questions:
1. Are there any intrinsic blockers to parallelizing the generated feature tests that execute in an autotools `./configure` script?
    - For example, I've been told that feature tests currently write to a single output file, which would get clobbered if we were to naively parallelize the test execution, but I was hoping that each test could be made to write to a temp file instead if that's true.
2. Which codebase (autoconf, automake, m4, ?) does the work of generating the script that executes tests in serial, and where in that codebase does this occur?
    - I've been perusing clones of the autoconf and automake codebases and I've been unable to locate the logic that actually executes each test in sequence.
3. How should we expose the option to execute tests in parallel?
    - In order to serve the purpose of improving `./configure` invocation performance, we would probably want to avoid requiring an `autoreconf` (spack avoids executing `autoreconf` wherever possible).
    - Possibly an option `autoreconf --experimental-also-generate-parallel-tests`, which would enable the end user to execute `./configure --experimental-execute-parallel-tests`?

Please feel free to link me to any existing content/discussions on this if I've missed them, or redirect me to another mailing list. I'm usually pretty good at figuring things out on my own but have been having some difficulty getting started here.

Thanks so much,
Danny





[Index of Archives]     [GCC Help]     [Kernel Discussion]     [RPM Discussion]     [Red Hat Development]     [Yosemite News]     [Linux USB]     [Samba]

  Powered by Linux