Adam Williamson said the following on 11/20/2009 08:37 PM Pacific Time:
> On Fri, 2009-11-20 at 15:43 -0800, John Poelstra wrote:
>> I've been been thinking for a while that it would benefit us to rework
>> our release criteria
>> (https://fedoraproject.org/wiki/Fedora_Release_Criteria) from being
just
>> a description of blocker bugs to more about the broader criteria that
>> needs to be met to issue a public release.
>
> I was talking to James Laska about this earlier today, and we both have
> thoughts on it too. I think it's a great idea to have revised and more
> detailed criteria, and I especially like the organization - a main page
> with an overview, and detailed criteria for each release.
>
> I had one major high-level suggestion. Release criteria are more or less
> unavoidably partially subjective. I don't think it's feasible to come up
> with concrete rules to cover every possible situation. Therefore, the
> criteria should explicitly embrace and cover this subjectivity. It
> should be made clear that things like 'boots successfully' are to some
> degree dependent on subjective, contextual judgements; do we block the
> Alpha for a bug that stops 0.1% of systems booting? 0.5%? 1%? 5%? I
> think rather than trying to define something like this, we should just
> explicitly acknowledge that it'll be a judgment call.
>
I agree that there will always be some level of subjectivity. The part
that has bothered me in the past was the notion that because there is a
level of subjectivity, defining things any more was impossible or a
waste of time and that people should just be okay with it. For the
example you've given above I think we could extend it to say "most
systems boot successfully" where 'most' becomes the part that is up for
a judgement call. Yes, I would agree that percentages for this example
would not be helpful. In other instances like "how many of the test
cases need to be run or pass"--a percentage of completion or success can
be useful depending on how it is written.
I agree that we can't come up with concrete rules to cover every
scenario, but as you suggest I think we can add a few more parameters to
explain the our thought process or be more explicit about where the
judgement calls are and who will make them.
I'm expecting that creating more detailed release criteria will be an
iterative process that we may not get right the first time and that is
okay. This is exactly how the current feature process came to maturity.
We tweaked and changed it over two or three releases to the point that
we rarely tweak it any more. But when we do need to tweak it there is a
larger framework to work within and we can make minor changes without
having to change the whole process.
This is why I think the additional page layout will help the release
criteria pages mature in the same way. The feature process still isn't
perfect, but it is worlds better than what we had before Fedora 8. I'm
advocating the same thing for our release criteria readiness
decisions... where each release things get a little more specific,
clearly documented and more widely understood by more people. This is
one way to scale and increase the chances of encouraging more new people
to participate in our community.
John
--
fedora-test-list mailing list
fedora-test-list@xxxxxxxxxx
To unsubscribe:
https://www.redhat.com/mailman/listinfo/fedora-test-list
