On Sun, Jan 24, 2021 at 7:54 PM Kevin Fenzi <kevin@xxxxxxxxx> wrote: > > On Sun, Jan 24, 2021 at 08:38:44PM +0100, Jakub Jelinek wrote: > > On Sat, Jan 23, 2021 at 03:21:14PM -0800, Kevin Fenzi wrote: > > > > I've been poking at it more today... > > > > > > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=60286478 > > > > > Total time 13:19:20 > > > > > Task time 1:11:53 > > > > > > > > Yeah, basically kojid get OOM killed, which causes it to restart and > > > > koji sees the build never finished and restarts it. ;( > > > > > > > > Right now I am going to take them all down and: > > > > > > > > * remove the qemu highmem=off (this is not needed in the lpae case for > > > > us, just non lpae, and might be causing some, but not all issues) > > > > > > > > * reduce their memory to 16GB. At 16GB I was able to get a python3.8 > > > > build cleanly with tests in 40min. Right now with 24GB on the builders > > > > they OOM kojid (as you see with gcc above). > > > > > > > > Hopefully those two things will make them more stable... > > > > > > ok. Done. All of them are up with 16gb and no highmem=off. > > > > > > I really hope this will stablize them before the mass rebuild. > > > > Total time 34:23:40 > > Task time 1:16:34 > > > > Already 5 times OOM killed. > > Yep. Been trying other things on it today in staging to get a stable > builder. :( > > I'm now trying (in staging) moving them back to direct boot from uefi. > If that doesn't work on f33, I will just punt and move them back to f32 > (like they were before) with a really old kernel. ;( maybe try the 5.6.x kernel on the uefi/f33 userspace, are any of the new oomd or related new fangle dangle OOM killers running on the builders? _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
