On Wed, Oct 20, 2021 at 03:06:21PM +0200, Jakub Jelinek wrote: > Hi! > > I've been trying to build gcc in rawhide and f35 for more than a week now, > but haven't succeeded due to instability of the armv7hl builders. > E.g. the > https://koji.fedoraproject.org/koji/taskinfo?taskID=77117620 > build succeeded on all but armv7hl in less than 22 hours (on aarch64 > even took less than 5 hours), but armv7hl kept rebooting or whatever > (impossible to find out from the logs) 8 times, see > https://koji.fedoraproject.org/koji/taskinfo?taskID=77117797 > Buildroots list. It's kojid being OOM killed. So, it restarts, sees that it's not running the build it's supposed to be, starts it over, gets OOM killed, repeat. > Eventually I've cancelled that build after 151 hours, disabled > LTO on armv7hl (for GCC bootstrap) and am building now > https://koji.fedoraproject.org/koji/taskinfo?taskID=77518894 > https://koji.fedoraproject.org/koji/taskinfo?taskID=77518968 > but those again show total time of ~ 21 hours, but armv7hl task times > of 1.5 and 10 hours, so the f35 build was retried already 2 times and f36 > once. > > Any recent kernel changes on the boxes, or is there any way to find out > what's going on with them, whether it oopses, OOMs or something else? Its OOM. It's basically this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1920183 It's much less common with the kernels we are running ( 5.12.19-300.fc34.armv7hl+lpae ) but it still does hit some packages. :( Nothing has changed on the builders. We are in final freeze and the only changes made on them are security updates, and none of those look like they would affect builds (vim, libjpeg-turbo, libvirt, openssh, curl) I can try f35 + latest kernel in stg and see if it's any better or this bug magically got fixed. :( > In late August the package built just fine and took ~ 27 hours to build on > armv7hl. Things should be like they were in late Aug. :( kevin
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
