Hey folks! So...just wanted to kind of put out there that there seems to be a lot of weird stuff happening in Rawhide right now. All different, but maybe, somehow, stemming from the same cause, or a few causes? Weirdness No. 1: rsync exiting uncleanly during live installs. We have a few different cases of this. One seems to be size related. In https://bugzilla.redhat.com/show_bug.cgi?id=2248759 there are problems with setting attributes. In https://bugzilla.redhat.com/show_bug.cgi?id=2329379 I found that rsync seems to exit 23 with no other obvious error message very often when running KDE and GNOME live installs of Rawhide; a recent anaconda change made it intentionally die when this happens, but it happens so often I've had to revert that anaconda change or else openQA tests almost never pass. Weirdness No. 2: UEFI live installs failing to boot past grub. This has been happening for the last few days. It looks like https://openqa.fedoraproject.org/tests/3059333#step/_do_install_and_reboot/115 . We get an apparently-normal install, then on reboot, the system dies at a grub prompt after trying to start shim. Weirdness No. 3: Weird build failures. There's a kernel build at https://koji.fedoraproject.org/koji/taskinfo?taskID=126333596 which failed with this message: arch/x86/tools/insn_decoder_test: error: malformed line 3566536: 2_>:ffffffff81bdd390 I'm almost sure I've seen the same thing in another build recently too. Weirdness No. 4: openQA tests failing in bizarre ways to do with filesystems. In https://openqa.fedoraproject.org/tests/3057221#step/_live_build/7 it couldn't write to /etc because it was somehow a read-only filesystem? If you examine the journal messages in https://openqa.fedoraproject.org/tests/3057221/logfile?filename=serial0.txt , you see that various other things seem to be read-only filesystems too: Nov 28 03:35:00 fedora systemd-random-seed[507]: 0;1;38:5:185m0;1;38:5:185mFailed to remove extended attribute, ignoring: Read-only file system Nov 28 03:35:01 fedora systemd-tmpfiles[557]: Setting access ACL "u::rwx,g::r-x,g:adm:r-x,g:wheel:r-x,m::r-x,o::r-x" on /var/log/journal failed: Read-only file system Nov 28 03:35:01 fedora systemd-tmpfiles[557]: Setting default ACL "u::rwx,g::rwx,g:tss:rwx,m::rwx,o::r-x" on /var/lib/tpm2-tss/system/keystore failed: Read-only file system and several more. https://openqa.fedoraproject.org/tests/3058836 is the same. I can't find it right now, but I saw a couple earlier today which failed because there was no such filesystem as /dev/vdc1 - *right after* the test had created it with sfdisk*. It's hard to shake the feeling these might all have some kinda common cause, but it's hard to think what. Maybe kernel 6.13? The "rsync failure" thing is reproducible at least back to the 20241121.n.0 iso. I might try pinpointing exactly when that started happening tomorrow. If anyone has any ideas about all this, it'd be great. Right now I'm just firing the retry cannon at openQA failures as fast as I can :( -- Adam Williamson (he/him/his) Fedora QA Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw@xxxxxxxxxxxxx https://www.happyassassin.net -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue