As we get closer to pushing out our first round of updates, we need to (re)visit our publishing policy.
Warren had persuaded me to agree with him that an extra level of testing needs to happen prior to a public release. Currently the process in full is:
1) file a bug with packages fixing a certain flaw 2) QA the package for source aspects and general build issues 3) push the package into updates-testing once two people give "PUBLISH" votes. 4) once in updates-testing, one test of package in full production is enough to release the package into testing.
We must be able to use our discretion, and do MORE testing on all distributions especially for the important packages. This probably means get GPG clearsigned "name-version-release works for me on RH7.3 after 1 hour of functionality testing. I also verified that the binary ldd output matches RH's package" from multiple people for packages like screen, but if apache needs upgrading even more feedback should be submitted since it is such a complex and important package.
Another good test is using ldd and compare the output of the binaries in RH's package and Legacy's package. It is entirely possible that Legacy's binary in updates-testing could be missing functionality since BuildRequires are missing.
http://www.fedora.us/pipermail/fedora-devel/2004-January/002526.html
"Careful - Missing BuildRequires for Reproducibility"
Read this thread for details about the many missing BuildRequires in many RH packages. Multiple packages currently in LEGACY QA have this problem and need to be fixed.
http://www.fedora.us/wiki/PendingRepository
fedora.us QA pending channel detailed on this page serves a similar need to Legacy's updates-testing channel, but fedora.us never had this particular problem of missing BuildRequires since we were strict on BuildRequires requirements from the beginning.
The whole "updates-testing" thing is the new wrinkle that Warren talked to me about. What I'm struggling with is how to handle the -testing aspects.
A) Is the updates-testing package signed, and if so, with what key?
FedoraLegacy.org eventually, when the build server is in production.
B) How is the package announced?
Has the template been finalized? Perhaps consider using sha1sum instead of md5sum since it is less susceptible to the birthday attack?
C) How can we verify that the tester is really testing the package?
That is why multiple testers are needed. One of the testers should be the packager of course. People that have submitted good testing results and especially found ERRORS in the past are to be trusted more than new people.
D) Since we release the package across multiple releases, how long do we wait on a specific release to be tested before releasing the rest of the packages?
Use best judgement. Wait for enough clearsigned feedback based upon the importance of a package to avoid regressions. Something like apache would need far more testing than something like screen. But NEVER push it too soon.
Warren
