Re: 32bit arm builder issues

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Jan 25, 2021 at 12:19:13PM +0000, Peter Robinson wrote:
> On Sun, Jan 24, 2021 at 8:55 PM Kevin Fenzi <kevin@xxxxxxxxx> wrote:
> >
> > On Sun, Jan 24, 2021 at 08:21:48PM +0000, Peter Robinson wrote:
> > > On Sun, Jan 24, 2021 at 7:54 PM Kevin Fenzi <kevin@xxxxxxxxx> wrote:
> > > >
> > > > On Sun, Jan 24, 2021 at 08:38:44PM +0100, Jakub Jelinek wrote:
> > > > > On Sat, Jan 23, 2021 at 03:21:14PM -0800, Kevin Fenzi wrote:
> > > > > > > I've been poking at it more today...
> > > > > > >
> > > > > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=60286478
> > > > > > > > Total time      13:19:20
> > > > > > > > Task time       1:11:53
> > > > > > >
> > > > > > > Yeah, basically kojid get OOM killed, which causes it to restart and
> > > > > > > koji sees the build never finished and restarts it. ;(
> > > > > > >
> > > > > > > Right now I am going to take them all down and:
> > > > > > >
> > > > > > > * remove the qemu highmem=off (this is not needed in the lpae case for
> > > > > > > us, just non lpae, and might be causing some, but not all issues)
> > > > > > >
> > > > > > > * reduce their memory to 16GB. At 16GB I was able to get a python3.8
> > > > > > > build cleanly with tests in 40min. Right now with 24GB on the builders
> > > > > > > they OOM kojid (as you see with gcc above).
> > > > > > >
> > > > > > > Hopefully those two things will make them more stable...
> > > > > >
> > > > > > ok. Done. All of them are up with 16gb and no highmem=off.
> > > > > >
> > > > > > I really hope this will stablize them before the mass rebuild.
> > > > >
> > > > > Total time    34:23:40
> > > > > Task time     1:16:34
> > > > >
> > > > > Already 5 times OOM killed.
> > > >
> > > > Yep. Been trying other things on it today in staging to get a stable
> > > > builder. :(
> > > >
> > > > I'm now trying (in staging) moving them back to direct boot from uefi.
> > > > If that doesn't work on f33, I will just punt and move them back to f32
> > > > (like they were before) with a really old kernel. ;(
> > >
> > > maybe try the 5.6.x kernel on the uefi/f33 userspace, are any of the
> >
> > Alas, they don't boot. :(
> 
> Strange as I've had vanilla F-33 booting UEFI on a few devices via the
> images we got composing before GA and ImageFactory was about to boot
> them for the compose.

Yeah, but that was likely newer kernel? 5.6 was GA for f32...
it's old at this point. :) 

Anyhow, I have reinstalled all the builders back to fedora 32 and
basically the config that they had before I upgraded them. :( 
This should hopefully be stable for the mass rebuild. 

In the mean time I have filed: 
https://bugzilla.redhat.com/show_bug.cgi?id=1920183
on the OOM kojid killing. Hopefully we can figure it out and then I can
move them up after we do. 

kevin

Attachment: signature.asc
Description: PGP signature

_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux