On Wed, Aug 06, 2008 at 03:35:45PM -0400, David Malcolm wrote: > I've been working on a new web-based tool for analyzing Fedora: rpmgrok > > It digests built RPMs, analysing the metadata and payload, and stores > the results in a database. There's a web UI for viewing the data, an > XML-RPC interface for querying it, and a command-line tool for using the > XML-RPC interface. > > I've got a prototype running on: > http://publictest7.fedoraproject.org/rpmgrok > > More info (e.g. source code) can be seen at > https://fedorahosted.org/rpmgrok > > The idea is to provide a new way for Fedora developers, testers, and > other enthusiasts to track various things across the entire > distribution, without having to have a full tree installed. It's > probably usable by other Linux distributions. I like the part with the "Errors and warnings from rpmlint". I can imagine that someone that's say an expert on "invalid-desktopfile" issues could now dig into this much easier. Very nice! > rpmgrok is Free/Open Source software (licensed under the LGPLv2.1) > > == What does it track? == > - all symbols in binaries/libraries, and the dependencies between > them, so that you can see e.g. exactly what calls a particular function. > This can also be used to locate instances of static linkage. See e.g. > http://publictest7.fedoraproject.org/rpmgrok/elffile/258085 (details > of /lib/libexpat.so.1.5.2 from a built RPM) > - manifests of all RPMs, so that you can browse the files in packages > via a web UI. See http://publictest7.fedoraproject.org/rpmgrok/files > The file view is only interesting at the moment for ELF files (binaries) > and for .desktop files. > - all shared objects names, and the dependencies between them. See > e.g. > - http://publictest7.fedoraproject.org/rpmgrok/sonames (browsable > view of all sonames in the distro) > - http://publictest7.fedoraproject.org/rpmgrok/elffile/739167 > (details of /usr/lib/libxml2.so.2.6.32 from within a built libxml2 rpm) > - Everything implementing or linking against libpcre.so.0 down to > the level of individual binaries: > http://publictest7.fedoraproject.org/rpmgrok/soname/libpcre.so.0 > - results of rpmlint of all rpms. See > http://publictest7.fedoraproject.org/rpmgrok/rpmlint_messages for a UI > to browse by error message, and e.g. > http://publictest7.fedoraproject.org/rpmgrok/rpmlint/dangerous-command-in-%25post for an example of all error messages of a particular kind. It may be worth fixing some rpmlint errors (though others look like false positives, and others are probably not worth it) > - all .desktop files and their fields so that you can e.g. find > applications that can handle PDF files. See e.g. > http://publictest7.fedoraproject.org/rpmgrok/mimetype/application/pdf > for a view of all desktop files that can handle "application/pdf", and > e.g. http://publictest7.fedoraproject.org/rpmgrok/desktopfile/253580 > showing a specific desktop file > - SLOCcount stats (see http://www.dwheeler.com/sloccount/ ) for > prepped source trees (e.g "what % of Fedora is in C/C++/Python?" etc). > Don't have the data prepped yet. > - any other kind of thing we want to add (provided there's a sane way > to gather it in a script and slurp it into the database, of course...) > - sizes of packages; why is package foo so big? > - report on all fonts in the distro, and what packages provide them > etc > > Note that due to my poor css there are lots of links that don't show up > as such in the various table views. You may need to explore with the > mouse to find all of the cross-referencing that the web UI has. > > == What's it currently showing? == > I queued up an analysis of all of rawhide as of 2008-07-25 on i386; a > little over 10000 built packages. It took about a week to process, and > about 200 of these jobs failed for one reason or another. See > https://fedorahosted.org/rpmgrok/ticket/9 for more info. > > So the db is currently just showing a snapshot in time of rawhide two > weeks ago, on one architecture (and missing 2% of the packages due to > errors). > > Ultimately I want to build things up so that we can show time-based > trend reports e.g. the size of a minimal install over time (or > whatever). > > == Help Needed! == > Hopefully this looks of interest to people. > > I need help with coding, with sysadmin work, with making the UI better, > and with things I probably haven't thought of yet etc. I hope this can > be a useful tool for Fedora. > > If you're interested in hacking on rpmgrok, get in touch. The README > file is hopefully of interest; see > http://git.fedorahosted.org/git/rpmgrok.git?p=rpmgrok.git;a=blob_plain;f=README.txt;hb=HEAD README.txt > > It's implemented using TurboGears and SQLAlchemy (specifically, > sqlalchemy 0.4, since it uses polymorphic inheritance features from that > version). > > It also has a somewhat general-purpose task scheduler, used to control a > pool of worker hosts that do the actual analysis. It ought to be > pluggable to do other types of task. > > == Source Code == > Git URLS are: > git://git.fedorahosted.org/rpmgrok.git > ssh://git.fedorahosted.org/git/rpmgrok.git > http://git.fedorahosted.org/git/rpmgrok.git > (you need to be in the gitrpmgrok of the Fedora Accounts System to have > git push privileges; talk to me if you want to get involved) > > == Related work == > Inspiration includes > - the OpenGrok project (see > http://opensolaris.org/os/project/opengrok/ though that appears to focus > on source trees, whereas rpmgrok focuses on built packages) > - the Debian project's Lintian tool (see http://lintian.debian.org ) > > Enjoy! > Dave > -- Axel.Thimm at ATrpms.net
Attachment:
pgpVKoJ1LnLS6.pgp
Description: PGP signature
-- fedora-devel-list mailing list fedora-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-devel-list
