CI for Bodhi updates has a bunch of tests like fedora-ci.koji-build.tier0.functional fedora-ci.koji-build./plans/public.functional that routinely fail. In fact, they are designed in a way where they _must_ fail for many important packages: systemd, fedora-release, coreutils, and anything else which has conflicting subpackages. (Example: https://bodhi.fedoraproject.org/updates/FEDORA-2024-3aafcac6a8) This is particularly annoying for multi-package updates, because the test will fail if _any_ of the packages are like this. Package installability is the most basic test. This should never fail. Can we please make this not fail? Or if it's not possible to make them work, disable them? In the current state we just waste everybody's time and teach packagers to ignore CI results. I'm looking for a solution which doesn't just skip the installability tests altogether. For zuul, we can set install_repo_exclude with a list of subpackage names in .zuul.yaml. That's an acceptable solution, but this configuration should not be specific to zuul, but should be picked up automatically by everything that tries to install the build outputs. (Maybe, in some distant future, maybe even 'mock --postinstall' could understand it? That currently fails for those packages in exactly the same way as the CI tests…) (Ideally, it would be possible to test installation of all subpackages. Those excluded subpackages may have bugs like any other package. A solution where we can specify which packages to install in groups would be great. But I guess that's hard. I'd settle for a half-solution like with zuul.) -- A second problem is that when the tests fail, it's just soooo hard to find out why they failed. From the bodhi status page, one has to go to the Jenkins status page, guess that it's useful to look at Console Output, scroll over a few pages of incomrehensible JSON gibberish, guess that it's worth clicking on Testing Farm Artifacts URL, click that, scroll pages of output to see "guest setup failed: Test environment installation failed: Install packages". Why? Oh, maybe it's time to click on random links. And yes, under "build installation" there is 80 links to text files, and one of them (60-Install-packages.txt) shows: Failed to resolve the transaction: Problem 1: package coreutils-9.5-4.fc41.x86_64 from @commandline conflicts with coreutils-single provided by coreutils-single-9.5-4.fc41.x86_64 from @commandline - conflicting requests ... Maybe if somebody is using the CI every day, they can memorize those magical pathways, but for a Joe Random Packager like me, it's takes way more time than it should to find the actual failure. Right now we show the debug logs for the CI pipelines and hide the actual test output. Can we instead make those 10 lines of actual test failure immediately visibile? -- A third problem is that the output is full of things that _look_ like errors: Warning: Permanently added '3.14.151.22' (ED25519) to the list of known hosts. egrep: warning: egrep is obsolescent; using grep -E <string>:36: (WARNING/2) Bullet list ends without a blank line; unexpected unindent. <string>:9: (WARNING/2) Definition list ends without a blank line; unexpected unindent. This makes those jobs look semi-abandonded. It also makes it hard to look for the real error, because one has to consider each of those lines and answer the question "is this the error I'm looking for?". Again, a time drain and a source of confusion for packagers who don't use the system every day. Zbyszek -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
