Hello,
po 15. 7. 2024 o 10:57 Zbigniew Jędrzejewski-Szmek <zbyszek@xxxxxxxxx> napísal(a):
On Mon, Jul 15, 2024 at 10:39:37AM +0200, Cristian Le via devel wrote:
> Hi Zbyszek,
>
> On 2024/07/14 20:04, Zbigniew Jędrzejewski-Szmek wrote:
> > I'm looking for a solution which doesn't just skip the installability
> > tests altogether.
>
> On PRs with zuul or FedoraCI automation, the same instability tests that are
> done for Bodhi are performed. But what would help is to make these tests as
> required to pass unless they are manually waved. Manually that can be done
> by setting `gating.yaml`. There was some discussion on making some of these
> tests as gating by default.
>
> Another issue specific to installability is that the issue is often further
> down the stream, particularly with the SELinux test. Definitely these need
> to be tracked down and fixed.
I fully agree. But a test that just does 'dnf install rpms-from-update/*.rpm'
will predicatably fail.
> > A second problem is that when the tests fail, it's just soooo hard to
> > find out why they failed.From the bodhi status page, one has to
> > go to the Jenkins status page, guess that it's useful to look at
> > Console Output, scroll over a few pages of incomrehensible JSON
> > gibberish, guess that it's worth clicking on Testing Farm Artifacts URL,
> > click that, scroll pages of output to see
> > "guest setup failed: Test environment installation failed: Install packages".
>
> Weird, when the test is finished, you should have only the final
> testing-farm results page. Here's an example [1]. Maybe in your case it
> encountered an internal failure?
>
> [1]: https://bodhi.fedoraproject.org/updates/FEDORA-2024-57f489c90d
Maybe I'm doing things wrong. I'd be happy to learn.
I do the following:
1. Look at the bodhi update page (https://bodhi.fedoraproject.org/updates/FEDORA-2024-3aafcac6a8)
2. Click on 'Automated Tests'
(There seems to be no URL for the view. This means that it's always
and extra click after every page load or reload.)
3. I click on one of the pinkish lines, e.g. the first one.
(Another usability problem here is that the click open a new page in
new tab/window. Why, oh why? I want to use left-click to open a link
in the existing tab, and middle-click to open a new tab. The current
UI breaks navigation.)
4. I switch to the new tab and see
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/398487/
(BTW, I mentioned unrelated scary-looking warnings in my OP.
Here:
The following steps that have been detected may have insecure
interpolation of sensitive variables (click here for an explanation):
httpRequest: [TESTING_FARM_API_KEY]
)
5. Click on 'Console Output'
6. Click on 'Testing Farm Artifacts URL: https://artifacts.dev.testing-farm.io/25316385-50d8-42b4-b4c1-3eff059034eb'
7. Click on 'build installation' (https://artifacts.dev.testing-farm.io/25316385-50d8-42b4-b4c1-3eff059034eb/guest-setup-09b7edc6-0b7e-431b-ae68-afac2527fbb1/artifact-installation-09b7edc6-0b7e-431b-ae68-afac2527fbb1)
8. Click on 60-Install-packages.txt
So 8 steps to get to the actual result…
This is not actually the installability test. This is a functional test where Testing Farm couldn't prepare the machine and thus the actual test didn't even start.
But it's true that figuring out what went wrong and where is an impossible task for mere mortals.
Thanks,
Michal
Zbyszek
--
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
-- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
