Every twelve months or so there is a new major release of perl and we do a mass rebuild of all perl packages. Last time, for the perl 5.14.x release, an attempt was made to do a clean bootstrap of the Fedora perl ecosystem, starting with the main perl package and then incrementally building other modules on top of that, avoiding pulling in packages built with the old perl 5.12.x release. However, this was a struggle because of widespread circular build dependencies between packages. In order to work around this, a %perl_bootstrap macro was defined, which packages could check to alter their build and/or runtime dependencies so as to break the cycles and allow a clean bootstrap. Unfortunately though it all took too long to locate and fix the cycles and we ended up pulling in old 5.12.x builds to get the last few hundred packages built. As it will soon be time to do a mass rebuild for perl 5.16.x, and similar bootstrapping issues affect the secondary architectures when trying to get their first Fedora build up, I decided to try to work through all of these issues ahead of time, find and fix the dependency cycles and work out an optimum build order for the whole Fedora perl ecosystem. Here is what I came up with. First of all, a high-level overview of my approach. I split the Fedora perl ecosystem into two groups of packages, which I've called "providers" and "consumers". The "provider" packages are those that contain perl modules in the %perl_vendorlib/arch and %perl_perllib/installarch directories, and those are the ones that I have concentrated on as they all need to be rebuilt. The "consumer" packages are those that have dependencies on the "provider" packages but aren't actually "providers" themselves. I don't think they really need rebuilding, though doing so should be trivially achieved in a single pass once the "providers" have been done. The "consumers" do need to be taken into account during the bootstrapping process through, because they (e.g. automake/autoconf) may be needed to build some of the "provider" packages. I've taken as the start point of the process the point at which the main perl package has been built and all of the packages in the mock buildroot are installable. Thereafter, I've split the bootstrapping process into a series of passes, where I first determine which of the "consumer" packages have become available (a large proportion only require Perl core modules and can installed as soon as the main perl package has been built at the start of the process), and then which of the "provider" packages can be built, which means that all of their build requirements (with %perl_bootstrap assumed to be defined to 1) are available, plus all of the dependencies of those build-requirements. I have assumed that all of the dual-lived modules are available from the start of the process, so there's no need to wait, for instance, for perl-Module-Build to be built before perl(Module::Build) is available for building against. All of the packages built in any given pass should be buildable in parallel, and will have at least one build dependency that was built in the previous pass. The result of this process is available at http://www.city-fan.org/~paul/perl-boot/buildorder The main bootstrap process takes 38 passes, with perl-Task-Catalyst the last "provider" package built at the top of the dependency tree. Following the bootstrap build, I feel there should be three further sets of rebuilds with %perl_bootstrap undefined: 1. Packages that have different runtime dependencies when built with %perl_bootstrap defined. There are mercifully few of these and they need to be done first to get to a state where all of the packages in the repo are available with the same dependency tree as if they'd all been built without special bootstrapping treatment. 2. Packages that have different build requirements when built with %perl_bootstrap defined. These will typically have run fewer tests in %check than they would have done for a regular build, so rebuilding them checks for FTBFS issues around the extra tests. It's also possible that the change in the runtime dependency tree resulting from the rebuilds in the previous pass might result in extra packages been pulled into the build too, which could affect the build. 3. Packages that were not rebuilt in the second post-bootstrap pass but whose buildroots would have had different packages as a result of the changes in the runtime dependency tree from the first post-bootstrap rebuild pass. For example, perl-Bio-SamTools would have been built without perl-bioperl-run in its buildroot originally but a rebuild would pull it in via the perl-bioperl<->perl-bioperl-run circular dependency (I wouldn't be surprised actually if the bootstrap build of perl-Bio-SamTools failed because of this but I haven't check that yet). Again, this is a check for FTBFS issues. If it's felt that the "consumer" packages should be rebuilt too, that could be done at this stage. I thought about describing my approach to working all of this out too but I think this email is long enough already. I can discuss that in follow-up messages if anyone is interested or would like to try/modify/improve it themselves. Paul. -- Fedora Extras Perl SIG http://www.fedoraproject.org/wiki/Extras/SIGs/Perl perl-devel mailing list perl-devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/perl-devel
