On Tue, 2025-02-25 at 15:29 +0100, Petr Pisar wrote: > V Tue, Feb 25, 2025 at 02:50:53PM > +0100, mkolman@xxxxxxxxxx napsal(a): > > On Thu, 2025-02-20 at 18:34 +0000, Richard Hughes via devel wrote: > > > Hi all, > > > > > > My CI has been failing for the last 24h with: > > > > > > [ 4/48] Installing fwupd-0:2.0.7-0.1alpha.fc41.x86_64 100% | > > > 39.9 > > > MiB > > > terminate called after throwing an instance of > > > 'std::length_error' > > > what(): basic_string::_M_replace_aux > > > contrib/ci/fedora-test.sh: line 3: 7 Aborted > > > (core dumped) dnf install -y dist/*.rpm > > > > > > Has anyone else seen this? Frustratingly I can't reproduce > > > locally. > > We have been seeing it sporadically in the Anaconda CI in the last > > few > > weeks, in just a single specific test workflow (that validates > > Anaconda > > RPMs have been built correctly) - I failed run looks like this: > > > > https://github.com/rhinstaller/anaconda/actions/runs/13498853227/job/37712147166?pr=6202 > > > > > > [ 1/323] Verify package files 0% | 0.0 B > > [ 1/323] Verify package files 21% | 69.0 B > > [ 1/323] Verify package files 34% | 112.0 B > > [ 1/323] Verify package files 42% | 137.0 B > > [ 1/323] Verify package files 57% | 183.0 B > > [ 1/323] Verify package files 73% | 237.0 B > > [ 1/323] Verify package files 94% | 303.0 B > > [ 1/323] Verify package files 100% | 321.0 B > > [ 1/323] Verify package files 100% | 321.0 B > > [ 2/323] Prepare transaction 63% | 203.0 B > > [ 2/323] Prepare transaction 100% | 321.0 B > > [ 2/323] Prepare transaction 100% | 321.0 B > > [ 3/323] Installing libblockdev-utils-0:3.3.0- > > 99.20250213132035617502.master.0.gf5ed6a16.fc42.x86_64 0% | > > 0.0 B > > [ 3/323] Installing libblockdev-utils-0:3.3.0- > > 99.20250213132035617502.master.0.gf5ed6a16.fc42.x86_64 100% | 44.5 > > KiB > > [ 3/323] Installing libblockdev-utils-0:3.3.0- > > 99.20250213132035617502.master.0.gf5ed6a16.fc42.x86_64 100% | 44.5 > > KiB > > [ 4/323] Installing python3-gobject-base-0:3.50.0-3.fc42.x86_64 > > 100% | > > 1.5 MiB > > [ 4/323] Installing python3-gobject-base-0:3.50.0-3.fc42.x86_64 > > 100% | > > 1.5 MiB > > [ 5/323] Installing libnl3-0:3.11.0-3.fc42.x86_64 100% | 1.0 MiB > > [ 5/323] Installing libnl3-0:3.11.0-3.fc42.x86_64 100% | 1.0 MiB > > [ 6/323] Installing libmnl-0:1.0.5-7.fc42.x86_64 100% | 56.4 KiB > > [ 6/323] Installing libmnl-0:1.0.5-7.fc42.x86_64 100% | 56.4 KiB > > [ 7/323] Installing satyr-0:0.43-5.fc42.x86_64 100% | 345.6 > > KiBterminate called after throwing an instance of > > 'std::length_error' > > what(): basic_string::_M_replace_aux > > make: *** [Makefile.am:329: container-rpm-test] Error 134 > > > > > > Its definitely a race condition, as it sometimes happens, and > > sometimes > > not. We have also made a few other observations: > > * we have never seen this happen outside of the GitHub CI infra > > -> eq. doing 100-200 locally running RPM test runs did not trigger > > this > > * is most often happens with the satyr package, but rarely it > > crashed > > with a different package > > -> likely satyr is just the first package that does something that > > has > > a chance to trigger this > > * the chance of this occurring seems to fluctuate over time - some > > days > > its not reproducible at all, some days it happens all the time > > -> maybe its related to load in the GitHub infra or some other side > > effects ? > > > > Unfortunately the environment this is running in is rather > > barebones > > and under many layers of infrastructure, so we were not able to dig > > up > > some more debugging data or a crash dump so far from when it > > happens. > > :P > > > Could you gather how TTY is configured in that infrastructure, if > there is > any? E.g. executing "stty -a" before the crashing dnf command. Managed to inject it somehow: + echo 'AAA: stty output' AAA: stty output + stty -a speed 38400 baud; rows 0; columns 0; line = 0; intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0; -parenb -parodd -cmspar cs8 -hupcl -cstopb cread -clocal -crtscts -ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff -iuclc -ixany -imaxbel -iutf8 opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0 isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop - echoprt echoctl echoke -flusho -extproc + echo 'AAA: stty output - done' AAA: stty output - done Full run log: https://github.com/rhinstaller/anaconda/actions/runs/13526227989/job/37797366415?pr=6194#step:6:7208 Hope it was at the right place/environment, as I've mentioned, quite a few layers are in play. :P > > -- Petr -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
