Re: Help with systemd/cgroup task limits in koji

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




Am 21.02.23 um 07:41 schrieb Florian Weimer:
* Kevin Fenzi:

Greetings.

We are running into some anoying limits on koji builds of chromium.

First, since a long time ago, the koji.service file we are using has:

TasksMax=infinity

But yet, chromium was failing, seemingly hitting a task limit.
"ninja: fatal: posix_spawn: Resource temporarily unavailable"
in the build and:
"kernel: cgroup: fork rejected by pids controller in
/machine.slice/machine-7d12b2e6dcfb4230b04d2c2c0b499171.scope/payload"
on the builder.

Investigation and some help from folks in the #devel room
(many thanks glb!)
Showed that the systemd-nspawn container mock started has:

systemctl show systemd-nspawn@0b3f01a2a8e345a389b30c477812c471
TasksMax=16384

So, I put in place a:
/etc/systemd/system/systemd-nspawn@.service.d/override.conf
with:

[Service]
TasksMax=infinity

and that seemed to be used for the mock systemd-nspawn containers.

However, the builds with lots of cpus is now failing later with:

Error: spawn /usr/bin/node-18 EAGAIN
     at Process.ChildProcess._handle.onexit
(node:internal/child_process:283:19)
     at onErrorNT (node:internal/child_process:476:16)
     at processTicksAndRejections (node:internal/process/task_queues:82:21)
[!] Error: unfinished hook action(s) on exit:

Is there yet another layer here that has another limit?

Is there anything here I can set that says "infinity all the way down' ?

Assistance welcome. I can file a systemd bug, but I am not sure
this is a bug more than a lack of documentation.
It could be an old kernel bug:

   Task exit is signaled before task resource deallocation, leading to
   bogus EAGAIN errors
   <https://bugzilla.kernel.org/show_bug.cgi?id=154011>

There have been recent namespace optimizations which introduce a similar
pattern there.  While they improve throughput in many cases, continuous
allocation and deallocation can now fail, even though the program logic
ensures that resources are never exceeded.

i am not sure if it's an old kernel bug, because kernel-6.1.7-200.fc37.aarch64 is running on koji builds of chromium.

Than
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux