On Tue, 2022-08-30 at 10:19 +0200, Petr Pisar wrote: > V Tue, Aug 30, 2022 at 09:39:27AM +0300, Alexander Bokovoy napsal(a): > > On ma, 29 elo 2022, Adam Williamson wrote: > > > On Tue, 2022-08-30 at 00:32 -0400, DJ Delorie wrote: > > > > It sounds to me like the problem is "how do we best use the available > > > > automated test resources?" so I'll answer accordingly. Ignore me if I > > > > misunderstood ;-) > > > > > > No, not really, sorry if I didn't explain clearly enough :D It's more > > > just a 'what's the best way to work what I want to do around the > > > existing definitions' question. What I want to do is have updates of > > > FreeIPA-related packages gated. The awkwardness is that right now the > > > definition of what gets gated is "critical path packages", and those > > > clearly don't meet the current definition of "critical path". > > > > Not all of them meet the current definition of 'critical path' but many > > do. In fact, almost all crypto-implementing packages should be on that > > list, if not already. > > > > Ideally, use of gating tests should be defined in the component itself. > > Do we have a mechanism to trigger OpenQA runs from the gating.yaml in > > the specific component? If we do, then we really should move it there. > > > > Another part of the problem is that most of these runs should probably > > be consulting, not gating in the sense that failing them should give you > > a clue that things might be wrong but there needs to be a human overview > > of the failures. It is what we have right now with OpenQA-reported > > failures for Bodhi updates in most cases, the test results aren't really > > preventing the submissions from the going forward unless a maintainer > > defined the update configuration to be so. > > > > Ideally, we should have reverse dependency updates to trigger FreeIPA > > tests and give their results a visibility but don't block on failures. > > Exactly. If a package needs gating, it can enable it in its dist-git. At the > same time I agree with Kevin, that if a distribution/edition needs to > gate a set of packages, then it should also have it's own way to define the > set. I included this as an option in my initial mail and explained why I don't like it much: it's a lot of maintenance, and openQA test gating is kind of specifically conceived as being distro-wide. > > If a package needs to run tests from a group of related packages, the package > can either manually define the group, or there should be an out-of-band way of > defining the group. (TMT calls it a test plan.) This is not what we're talking about. openQA tests updates, not packages. The bundling is expected to be done at the update level. If the update does not include all the packages it should, that's a packager error and openQA tests failing is intended, valid and useful. > A reverse dependency is a possible, IMHO good, way of the definition. Alas, > Fedora TMT does not support it > <https://pagure.io/fedora-ci/general/issue/251>. > > I feel Adam is aware of it and tries to work around this TMT deficiency > with an allowedlist. This has nothing to do with TMT or Fedora CI in general (unless you count gating.yaml and Greenwave as part of "Fedora CI"). The openQA scheduler is an entirely separate codebase[0] and does not use TMT or anything else related to Fedora CI. In general, please remember that Fedora CI is not the only test system using the Bodhi/greenwave industrial complex. :D [0]: https://pagure.io/fedora-qa/fedora_openqa -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
