Re: Help with systemd/cgroup task limits in koji

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



* Kevin Fenzi:

> Greetings.
>
> We are running into some anoying limits on koji builds of chromium.
>
> First, since a long time ago, the koji.service file we are using has:
>
> TasksMax=infinity
>
> But yet, chromium was failing, seemingly hitting a task limit.
> "ninja: fatal: posix_spawn: Resource temporarily unavailable"
> in the build and:
> "kernel: cgroup: fork rejected by pids controller in
> /machine.slice/machine-7d12b2e6dcfb4230b04d2c2c0b499171.scope/payload"
> on the builder.
>
> Investigation and some help from folks in the #devel room
> (many thanks glb!)
> Showed that the systemd-nspawn container mock started has:
>
> systemctl show systemd-nspawn@0b3f01a2a8e345a389b30c477812c471
> TasksMax=16384
>
> So, I put in place a:
> /etc/systemd/system/systemd-nspawn@.service.d/override.conf
> with:
>
> [Service]
> TasksMax=infinity
>
> and that seemed to be used for the mock systemd-nspawn containers.
>
> However, the builds with lots of cpus is now failing later with:
>
> Error: spawn /usr/bin/node-18 EAGAIN
>     at Process.ChildProcess._handle.onexit
> (node:internal/child_process:283:19)
>     at onErrorNT (node:internal/child_process:476:16)
>     at processTicksAndRejections (node:internal/process/task_queues:82:21)
> [!] Error: unfinished hook action(s) on exit:
>
> Is there yet another layer here that has another limit?
>
> Is there anything here I can set that says "infinity all the way down' ?
>
> Assistance welcome. I can file a systemd bug, but I am not sure
> this is a bug more than a lack of documentation.

It could be an old kernel bug:

  Task exit is signaled before task resource deallocation, leading to
  bogus EAGAIN errors
  <https://bugzilla.kernel.org/show_bug.cgi?id=154011>

There have been recent namespace optimizations which introduce a similar
pattern there.  While they improve throughput in many cases, continuous
allocation and deallocation can now fail, even though the program logic
ensures that resources are never exceeded.

Guiseppe, any suggestions how to debug this?

Thanks,
Florian
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux