On Wed, 2024-02-28 at 09:03 -0800, Adam Williamson wrote: > Hey folks! Just an update on an issue I know some packagers are running > into. > > In the last few days, some Fedora CI jobs have been getting stuck and > timing out. Packagers will usually experience this as a test result > showing as "ABSENT" in Bodhi, e.g. for this update: > > https://bodhi.fedoraproject.org/updates/FEDORA-2024-ce1fb5a0db > > the fedora-ci.koji-build.tier0.functional result shows as ABSENT (red > line, question mark icon which shows ABSENT if you hover over it). > > We don't have any CI tests that are universally gating by default, but > some packages have opted in to gating on some CI tests, and some of > these have been stuck in updates-testing for a few days due to this > problem. > > It seems the issue was caused by a kernel 6.8 bug which broke quite a > lot of stuff last week - see > https://lore.kernel.org/all/6a150ddd-3267-4f89-81bd-6807700c57c1@xxxxxxxxxx/ > . The fix for that bug was in kernel 6.8rc6, which is now in both > Fedora 40 and Rawhide. Miroslav Vadkerti is aware of this and is > working to try and clean things up (update the base images for Fedora > CI and get all the timed-out tests re-run). I don't know what the ETA > is, but just wanted to let folks know it's at least being worked on. Update on this: turns out the kernel bug wasn't the main thing, that was causing some test errors but the main problem was that reporting of Fedora CI results to resultsdb simply broke on Saturday and nobody spotted that was the problem till today. We got that fixed today, and I have hacked up and am now running a script which should report all the results that were missed (hopefully correctly, and hopefully without overriding or duplicating or missing anything). Once it's finished running I'll do a sweep for updates still stuck in updates-testing and see where we are. -- Adam Williamson (he/him/his) Fedora QA Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw@xxxxxxxxxxxxx https://www.happyassassin.net -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
