[Base] Proposal for buildrequires cleanup janitorial initiative

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi everyone.

During last weeks Base WG discussion about package set and self hosting of Base we came to a point where especially the self hosting of Base would currently look absurd as we'd require more than 2000 components to do so.

For that reason we'd like to propose the following initiative:

Initiate build requires cleanup for base related packages in Fedora working with maintainers and the community. The goal is to reduce the number of self-hosting packages required for Base from currently over 2000 packages.

To kickstart this off one idea would be to start looking at very small self contained package sets that already exist:

https://fedoraproject.org/wiki/Base/Packages

This is a page thatHarald Hoyer put together and consists of a very limited set of packages that are together dependency complete.

Now, to figure our how the build chains for these packages look like i've cobbled together a (really bad) hack using python and repoclosure that basically takes a set of packages as an input (actually a set of requirements) and then spits out consecutively the groups of packages needed to build the previous ones, so basically a reverse grouped build order:

------------------ snipp here ------------------

import sys
import subprocess

def getBuildRequires(pkgset):
    requires = set()
cmd = ["repoquery", "--enablerepo=fedora-source", "--requires", "--srpm", "--source"]
    cmd.extend(pkgset)
    out = subprocess.check_output(cmd)
    for r in out.split("\n"):
        requires.add(r)
    return requires

def getSRPMs(requires):
    newpkgset = set()
    cmd = ["repoquery", "--whatprovides", "--source"]
    cmd.extend(requires)
    out = subprocess.check_output(cmd)
    for prov in out.split("\n"):
        if prov[-8:] != ".src.rpm":
            continue
        newpkgset.add(prov[:-8])
    return newpkgset

oldpkgset = set()
newpkgset = set(getSRPMs(sys.argv[1:]))

while len(newpkgset - oldpkgset) > 0:
    diff = newpkgset - oldpkgset
    print len(diff), diff
    oldpkgset = set(newpkgset)
    newpkgset = newpkgset | getSRPMs(getBuildRequires(diff))

print len(newpkgset), newpkgset

------------------ snipp here ------------------

The output is awful, but can be easily adapted to what one needs. It needs to be run on a current Fedora system (F20 or rawhide) like so:

python bo2.py <componentname> *[<componentname>]

and it'll spit the different sets of package srpm NEVRAs that need to be built with the last output being the whole set of packages and the second to last one being the first component group that needs to be built. So if you only want the groups, just put a comment before the last line.

I know this could probably be written much nicer using the yum API, but it does the job for me for now and it's relatively fast (couple of minutes for a set of packages).

I'll add some better formated lists to Harald's wiki page so the buildreqs and groups can be studied and tackled by anyone who feels like wanting to help here. :)

And obviously if you as a package maintainer feel like you'd like to cleaning up your package, hopefully this tool will help you determine why the heck your package might have some really oddball buildrequire chain, too. :)

Thanks & regards, Phil

--
Philipp Knirsch              | Tel.:  +49-711-96437-470
Manager Core Services        | Fax.:  +49-711-96437-111
Red Hat GmbH                 | Email: Phil Knirsch <pknirsch@xxxxxxxxxx>
Wankelstrasse 5              | Web:   http://www.redhat.com/
D-70563 Stuttgart, Germany
--
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct





[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]
  Powered by Linux