Re: the sad state of installability tests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, 2024-07-15 at 08:45 +0000, Zbigniew Jędrzejewski-Szmek wrote:
> > Yes, I've opened many issues regarding Fedora CI. I think the problem is that
> > the CI maintainer is not a regular packager and thus does not observe CI
> > reults in real life. On the other hand, package maintainers do not have accces
> > to the CI system and need to rely on the CI people. Or a CI person?!. The head
> > count is probably another problem.
> 
> If there's one "CI person", then that is AdamW. He's doing great work
> (also in the update I linked in my original post). IIUC, AdamW's focus
> is on the 'update.*' tests, and those are fine, they generally pass.
> The bodhi results page says:
> 
>   For help debugging failed Fedora CI tests (fedora-ci.*), contact the Fedora CI team.
>   For help debugging failed Fedora CoreOS tests (coreos.*), contact the Fedora CoreOS team.
>   For help debugging failed openQA tests (update.*), contact the Fedora Quality team, ...
> 
> I added ci@xxxxxxxxxxxxxxxxxxxxxxx in CC.
> 
> Zbyszek

Just to be really clear about this: I am not personally responsible for
Fedora CI, and the Fedora QA team is not responsible for it as a group.
This is true at both Fedora and Red Hat levels. "Fedora CI" refers to
the Jenkins and Zuul pipelines that run the tests reported to resultsdb
with 'fedora-ci.' at the start of their name. I and my team *are*
responsible for openQA, which runs the tests reported with 'update.' at
the start of their name.

To be clear about something else - no "Fedora CI" test is gating for
any Fedora update unless a package has configured it to be so in its
per-package gating config. There is no project-wide gating on Fedora CI
tests. This is because we know none of them is reliable enough to gate
project-wide (and it's hard to read the results, and there's no one
person whose job it is to look at failures and help resolve them, and
so on). The only project-wide gating is on openQA tests, for critical
path updates.

Due to various...internal considerations, when the system called
"Fedora CI" was set up, it was intentionally set up under a different
team which is *not* the team I'm on. Resource constraints since then
have led to a situation where there really isn't anyone whose primary
full-time job is caring about Fedora CI, which is the reason a lot of
things to do with it are the way they are.

I don't love this situation, and I'd love to have the time and access
to work on improving the Fedora CI tests, but I just don't. Like the
team that is nominally responsible for it, I have a bunch of other
stuff that is considered higher priority. I have contributed a few
things to Fedora CI where I can (like the 'results dashboard' for the
installability test), but it's never the top of my to-do list, and I
don't have admin access to any of it. I'm just an outside contributor
to that effort like anyone else.

ci@ is indeed the right list for anything to do with Fedora CI.

We are aware of many of the problems with the installability test -
there's a ticket tracking them at
https://pagure.io/fedora-ci/general/issue/448 . There are lots of ideas
and plans there for improving things. The problem is just that nobody
ever gets a clear week to sit down and do them as their top priority
work. I've spent the last two months trying to clear my plate to work
on an idea which would necessarily involve some improvements to it and
I've never got there.
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw@xxxxxxxxxxxxx
https://www.happyassassin.net




-- 
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux