Re: Thoughts welcome: interface between automated test gating and the "critical path"

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]


On Wed, 2022-08-31 at 12:43 +0000, Zbigniew Jędrzejewski-Szmek wrote:
> On Tue, Aug 30, 2022 at 12:10:00PM -0700, Adam Williamson wrote:
> > On Tue, 2022-08-30 at 09:14 -0400, Ben Cotton wrote:
> > > From my perspective, anything that blocks the release is on the
> > > critical path. So any time there's a violation of the release criteria
> > > and the package is not on the critical path definition, that's a bug
> > > in the definition.
> > > 
> > > I recognize that this is a somewhat naïve view. For one, it may
> > > broaden the definition beyond the current capacity of our test
> > > infrastructure. It also may broaden the definition beyond what
> > > maintainers are willing to put up with. These are both legitimate
> > > problems. But the closer we can get to this ideal state, the better.
> > > 
> > > For anyone who is curious, I just searched for all accepted blockers
> > > in the "Fedora" product in Bugzilla. 327 components have been a
> > > blocker at least once. Some of those may no longer be blocking and
> > > others will be added over time as our criteria change. The full list
> > > with counts is at
> > > if
> > > you're interested.
> > 
> > Honestly, something along these lines would be my preference too, I
> > just don't know if others would agree/support changing the critical
> > path definition to "all release-blocking functionality" rather than
> > "functionality needed to boot a basically-functional system".
> I'd support increasing the scope to cover more packages in this fashion.
> Being on critpath list is somewhat annoying because the bodhi update
> minimum times are twice as long. For many packages this is a not a problem,
> popular packages get enough karma within a day or two,
> but if we expanded the list a lot, it could prove annoying to those
> packagers. I think we could discuss lowering the minium update time
> if this turns out to be the case.

So, that's an interesting question to consider as well, of course:
what's the actual impact of critpath?

It'd be an interesting outcome if we broadened the critpath definition
but diluted the Bodhi requirements, because the original purpose of
critpath was more or less entirely the stricter Bodhi requirements.
Until openQA came along, that was all it really meant: updates to these
packages have stricter Bodhi requirements.

Now, because I glued openQA to the critpath because it was handy, there
are two sets of consequences to a package being in critical path:

1. Tighter Bodhi requirements
2. openQA tests are run, and results gate the update (except Rawhide)

So, one of the implicit questions here is, is it OK to keep twinning
these two sets of consequences, or should we split them up? Splitting
them up kinda implies answer 2) from my original mail: "Keep the
current "critical path" concept but define a broader group
of "gated packages" somewhere". Because we would then need some new
concept that isn't "critical path". As I said, that's more *work* -
it'd require us to write new code in several places[0]. Even if we
decide it'd be nice to do this, is it nice *enough* to be worth doing
that work?

If we don't think it's worth doing that work, then we're kinda stuck
with openQA glomming onto the critpath definition to decide which
updates to test and gate, because I don't have any other current viable
choices for that, really. And we'd have to figure out a critpath
definition that's as viable as possible for both purposes.

BTW, one other thought I've had in relation to all this is that we
could enhance the current critpath definition somewhat. Right now, it's
built out of package groups in comps which are kinda topic-separated:
there's a critpath-kde, a critpath-gnome, a critpath-server, and so on.
But the generated critical path package list is a monolith: it doesn't
distinguish between a package that's on the GNOME critpath and a
package that's on the KDE critpath, you just get a big list of all
critpath packages. It might be nice if we actually did distinguish
between those - the critpath definition could keep track of which
critpath topic(s) a package is included in, and Bodhi could display
that information in the web UI and provide it via the API. That way
manual testers could get a bit more info on why a package is critpath
and what areas to test, and openQA could potentially target its test
runs to conserve resources a bit, though this might require a bit more
coding work on the gating stuff now I think about it.

[0]: at least: releng critpath generation scripts, bodhi, the openQA
scheduler, greenwave policies, and possibly greenwave
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha

test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct:
List Guidelines:
List Archives:
Do not reply to spam, report it:

[Index of Archives]     [Fedora Desktop]     [Fedora SELinux]     [Photo Sharing]     [Yosemite Forum]     [KDE Users]

  Powered by Linux