On 08/08/13 12:50, Petr Pisar wrote:
On 2013-08-08, Paul Howarth <paul@xxxxxxxxxxxx> wrote:
On 08/08/13 10:01, Phil Knirsch wrote:
Just like Seth's script it has a few obviously problems as for really
correct build ordering you need to flip-flop between srpms/specfiles and
binary rpms, but that will need a bit of a different approach. As it
stands right now some of the dependencies it can't detect and resolve
are build time generated ones, which might a showstopper for some uses
of it.
I've tested the version i have here with several complete past Fedora
trees, for texlive and KDE rebuilds and updates and for all the cases
the output look pretty sane.
Hope this helps,
We already know where it's beneficial to break build dependency cycles
and have built that into the spec files, triggered by the
%{perl_bootstrap} macro being set (which it is in the current Rawhide
perl). So a build ordering script for perl bootstrapping should take
that into account.
And therefore my slow script considers SRPM as well as binary packages
distinctively. Therefore it clones git repostories, creates SRPMs in mock
environment (because data from koji will differ from to-rebuild packages
due to the perl_bootstrap macro), retrieves just built binary packages,
extracts the requires/provides and solves them on its own. Of course it
submits whole groups of packages to rebuild and it waits for build root
rotation. It does prety much everything what is needed.
My own script does something similar but doesn't build the packages,
since there are hardly any for which the runtime dependencies of
bootstrapped packages are different from those of non-bootstrapped
packages - I have a separate override configuration file in which I
specify those differences. Needless to say, this saves a lot of time. So
the git checkout is used to generate SRPMs, and I get the buildreqs from
those, and using the existing Rawhide metadata for the runtime dependencies.
(Curriously, this year I could not build in 16 threads as last year,
becuse dist-git starts refusing on 10 threads. Also after ausil started
his mass rebuild, my koji client started getting host unreachable errors.)
However the really slow part is the dependency solver. I had no time to
improve it before this rebuild. I will do before next time. But now I'm
not going to change it becuse it's to complicated.
I did my depsolver in python, leveraging code from yum. It's actually
quite quick, much quicker than extracting the dependency data from git,
except in the presence of circular build dependencies, but it does a
decent job of identifying and prioritizing those, and I keep on top of
them by running the script periodically and raising bugs where needed.
Neverthless we are looming to the point where no new packages will be
possible to rebuild due to unsatisified depenencies. We are now at
95 % done. The rest will need fixing or dropping
<https://fedoraproject.org/wiki/Changes/perl5.18#Items_to_be_done>. Once
we get there, I will publish list of failed and unfinished packages
together with dependency graph.
I have already done 169 changes in the F20 packages to achieve those
95 % and I'm quite tired and sick of all of that. (Mainly
because maintainers do not upgrade their packages or because they do not
declare all needed dependencies). (I'm thinking of running dummy
rebuilds during a year regularly, to find all these buggy packages before
next Perl rebuild.)
It's a good plan. There are people running through all modules on CPAN
and raising bugs where they break with bleadperl but if upstreams don't
fix these bugs there's not much we can do.
Paul.
--
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
