On Mon, 07 Oct 2013 04:03:16 +0200
Ralf Corsepius <rc040203@xxxxxxxxxx> wrote:
> On 10/07/2013 03:49 AM, Tim Flink wrote:
> > On Sun, 06 Oct 2013 15:09:24 -0500
> > Rex Dieter <rdieter@xxxxxxxxxxxx> wrote:
> >
> >> Michael Schwendt wrote:
> >>
> >>> On Sun, 06 Oct 2013 19:11:41 +0200, Ralf Corsepius wrote:
> >>>
> >>>> I now see ... the version in f19 was greater than that in
> >>>> f20+rawhide, for whatever reasons.
> >>>> Actually, I wonder why AutoQA did not complain.
> >>> There are no AutoQA comments in that bodhi ticket at all. Almost
> >>> as if AutoQA has not been run for that update. Normally it would
> >>> add a comment also for PASSED tests.
> >> Is AutoQA enabled globally yet? Last I knew, it was an opt-in
> >> service.
> > AutoQA is run globally on all current fedora branches other than
> > rawhide but if maintainers want results emailed to them, it's
> > opt-in.
> >
> Bummer - Why? Isn't it AutoQA's job is to catch "must-not-happen"
> package bugs, i.e. to be *mandatory*?
The same old, tired excuse of time and priorities. Some of the pieces
needed to start gating builds or updates based on the output of
automated checks do not exist yet. Testing Fedora releases has a higher
priority than developing automation and most of the people interested
in and involved in automation development are also involved in the
testing process.
There are a couple of things that we'd need in order to start doing
that:
- support for excluding builds/updates during pushes
* I think that the capability to do this is mostly in place already -
the biggest thing it needs is a place to query before actually
moving things around.
- An official build/update gating policy (what checks need to pass
before a build moves into rawhide or an update moves into
updates-testing, how to handle overrides etc.)
- a method for overriding automated check results
* there is no such thing as a perfect automated check - there is
always the possibility of false negatives and false positives. If
we're talking about build/update gating, we need to have a way of
saying "no, you're wrong. this actually passed/failed"
Some more work on improving the checks themselves is needed but that's
a bit more on the minor side.
We have started working on plans to revamp our automation capabilities
but how quickly the work ends up happening happens will depend on
whether we can keep people working on development without slipping
releases and if the proposal to delay f21 for infrastructure work (qa,
releng maybe others) is accepted.
Tim
Attachment:
signature.asc
Description: PGP signature
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
