On 04/19/2017 09:16 AM, Daniel P. Berrange wrote:
On Wed, Apr 19, 2017 at 09:03:55AM -0600, Jim Fehlig wrote:
I recently received a report of a libvirt+Xen installation reaching the pids
cgroup controller TasksMax limit
kernel: [71282.213347] cgroup: fork rejected by pids controller in
/system.slice/libvirtd.service
The default setting of TaskMax is 512 on this system
# cat /sys/fs/cgroup/pids/system.slice/libvirtd.service/pids.max
512
Depending on domain type and configuration, I've noticed between 5-7 tasks
(IO threads, qemu process, etc.) are created when starting a domain. It
doesn't take too many domains before the 512 limit is reached.
LimitNOFILE was recently changed by commit 27cd763500 to support 4096
domains. Following similar logic, would it be ok to increase TasksMax to
32768? That would accommodate 4096 domains with 8 tasks each. TasksMax also
supports the special value of "infinity", but that seems a bit aggressive to
me.
KVM uses 1 thread per vCPU, so 8 pids is probably somewhat on the low
side if considering high vcpu counts. I think we could go for something
like 128k pids which allows for closer to 32 vcpus per guest
It looks like the vcpu threads are under
/sys/fs/cgroup/pids/machine.slice/machine-qemu<name>kvm.scope/, where pids.max
is already 16k. I don't think these would be constrained by attributes under
/sys/fs/cgroup/pids/system.slice/libvirtd.service. In fact, I don't see any
additional tasks under /sys/fs/cgroup/pids/system.slice/libvirtd.service/tasks
when starting a KVM guest.
Since the libxl library runs in libvirtd process, any threads/processes it
creates are accounted for in /sys/fs/cgroup/pids/system.slice/libvirtd.service.
We are probably safe with the more conservative 32k.
Regards,
Jim
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list
