Firstly, thanks for the 2.70 release! Yocto Project hasn't switched yet as whilst we tested it and reported issues, there was a last minute change which broke things for us. We'll get that sorted and upgrade. I am one of the lead developers of the Yocto Project and at its heart is OpenEmbedded which is basically a cross compiling operating system build tool. We primarily target Linux and specialise in allowing complete customisation of the end resulting images. We try and solve a complex problem, pulling together all the pieces of open source needed for an operating system including the toolchain, cross compiling them and allowing the user to customise them in infinitely different ways. Where the software uses autotools, we actually regenerate the configure scripts and makefiles from the source files before configuring and building the software. We do this since we may need to include arch support that isn't part of the original configure script, or to include fixes we carry to components like libtool, which I'm say to say doesn't work properly cross compiling out the box. One scary statistic is that if we look at where our builds spend time, 40% of our wall clock build time is spent in "do_configure" tasks, which is mostly running autotools and is single threaded. The compile takes much less time as we can do things in parallel. Of that 40%, about half of it is in autoreconf and half is running configure itself. I did profile configure and the problem is that its basically a fork bomb. Tons of forks to do trivial shell operations. I have less insight how to speed up the autoreconf side of things as that mostly seems to be in m4, although an m4 deletion party of obsolete things we don't care about has come to mind before. I like autotools, I've spent a lot of time debugging them but I also know they solve some hard problems. I do wish there were more active releases and an upstream we could engage with and get issues discussed and resolved somehow. I will observe that I once worked on a Makefile based build system (buildroot) and I then somehow ultimately became the bitbake maintainer which can be thought of as a custom python tool for parsing and dealing with metadata. Over the years I have come to love python for handling our recipes and configuration data. I saw suggestion of using some other language for configure than shell and the idea sounds intriguing, particularly after my positive experiences with python. In terms of CI, Yocto Project had a challenge there as we have huge possible numbers of configurations. Its taken us a few years but we do now have a CI system I'm quite proud of. In around 6 hours we can take a configuration and build it out from scratch for 4 principle architectures in 32, x32/n32 and 64 bit variants for multiple init systems, kernel versions, libc implementations and much much more. As an idea, here is a link to one of our test reports: https://autobuilder.yocto.io/pub/non-release/20210120-9/testresults/testresult-report.txt That does correctly say it attempted about 2.6 million tests. Included there are booting images under qemu for all our principle target architectures and running the test suites included with many pieces of software, including the testsuites with gcc, glibc, binutils, perl and python and the ltp suite. We also cover things like reproducibility. When we heard about the new release we did run 2.70 under this test matrix and reported issues back so hopefully the release benefited from this indirectly. Doing something simpler with build testing for autoconf and other GNU projects would be entirely possible, I mention this mainly to show what can be done! I do worry about the future for autotools, I see projects moving over to alternatives and I am happy to see the performance gains we get from it. I'd mention that the copyright assignment situation needed for contributions is prohibitive to people being able to participate (including me in the past). GNU's thoughts on version control and project interconnection is also somewhat discouraging to people in the wider open source ecosystem although I don't follow it closely. I've also not seen mention of it but the fact that GCC and the toolchain use an ancient version of autoconf has always been rather sad to me. I do think autotools has a proud heritage but to survive into the next decade, I do think it needs to change direction, stop supporting old obsolete systems and embrace more new technology, much like it would have done originally. Reducing to the lowest common denominator is ok for a while but ultimately signs a death warrant as I've seen observed on the libtool list by devlopers in recent years too. Everything here is meant as constructive support of trying to find a way to allow autotools to succeed into the future. Cheers, Richard
