On Fri, 06 Feb 2004 02:14:19 +0100, Jonas Pasche wrote: > I have put together a page that might help people doing QA: > > http://jonaspasche.de/fedoralegacy/qa-howto.html In such a detailed form this goes into a wrong direction IMHO. Some comments following... but first: Disclaimer in front: This is no attempt at stopping anyone from trying to create a "how to do QA" document, although I think there's no general recipe on how to do QA. If there are people who find such documents useful and if it helps them actually to get started with package reviews and approvals, I shut up. At this time, however, I'd like to avoid that the "QA hurdle" is set too high or that it is considered as set too high. Apart from that, copying some things from the fedora.us checklist simply doesn't make any sense. I like to point out that people who are new to reviewing packages should follow a top-down approach to testing packages (and build teams with people who look a level lower) instead of a bottom-up approach which starts with low-level technical package reviews: If Fedora Legacy wants to provide updated packages which fix security issues or critical bugs, the primary objective is to provide packages which fix those issues and which don't break anything else. This requires testing the built binary packages. If no binary packages are provided, it may be necessary to rebuild the source rpms. Like Eric Rostetter's page, an introduction to testing packages should start with stuff like how to rpmbuild a src.rpm as non-root user, how to extract a src.rpm, how to create an rpm diff against the previous src.rpm, and maybe later continue with additional checks at the level of the source rpm or binary rpm (e.g. ldd, rpm -qlvp and rpm -qR checks), meeting the requirements of a build system (=> completing the build dependencies). However, Fedora Legacy aims at modifying the source rpms as little as necessary, so reviewers don't generally need to spend any time reading the spec file beyond the diff against the previous release. Fedora Legacy has different requirements than Fedora.us. Such a checklist gives a false impression of what kind of package reviews are needed. For instance, the Fedora.us QA checklist has been misunderstood by people repeatedly as it covers many low-level issues but doesn't spend a single word on how much "in production" testing is required or when a package is classified as "stable" instead of "testing" or "unstable", respectively. A checklist makes the package verification procedure appear overly complex, complicated [and maybe even pedantic]. Or people go through the list step by step and don't examine anything else. Fedora.us has different requirements because it deals with entirely new package build scripts, which are often written from scratch or sometimes derived from other distributions. Sometimes a packager doesn't even take the time to understand what is done in the package build script and whether it does what it is supposed to do. Such circumstances require a different level of reviewing package submissions. Quite some of the Fedora.us guidelines and policies have been created, so the initial repository is filled with a solid base of usable packages, which other packages can built upon and which also serve as examples and inspiration, and to avoid that the repository is turned into a dumping ground for a wild mixture of packages, which would probably require increased maintenance upon updates and/or distribution upgrades (e.g. even improperly set dependencies can break a repository easily). Other requirements and procedures (e.g. different levels of trust and low-level package reviews) are because in an open community project you meet and work with people you don't know or haven't worked with before (do you build from src.rpm without comparing the package contents with its previous release?). Many of the steps in the linked/copied fedora.us checklist don't apply at all to Fedora Legacy. I feel that starting with this list as a basis is approaching the "QA problem" from the wrong direction. Start at the top and refine the procedures and policies. --
Attachment:
pgp00114.pgp
Description: PGP signature
