On Wed, Apr 19, 2017 at 01:11:41PM -0600, Jim Fehlig wrote: > On 04/19/2017 09:16 AM, Daniel P. Berrange wrote: > > On Wed, Apr 19, 2017 at 09:03:55AM -0600, Jim Fehlig wrote: > > > I recently received a report of a libvirt+Xen installation reaching the pids > > > cgroup controller TasksMax limit > > > > > > kernel: [71282.213347] cgroup: fork rejected by pids controller in > > > /system.slice/libvirtd.service > > > > > > The default setting of TaskMax is 512 on this system > > > > > > # cat /sys/fs/cgroup/pids/system.slice/libvirtd.service/pids.max > > > 512 > > > > > > Depending on domain type and configuration, I've noticed between 5-7 tasks > > > (IO threads, qemu process, etc.) are created when starting a domain. It > > > doesn't take too many domains before the 512 limit is reached. > > > > > > LimitNOFILE was recently changed by commit 27cd763500 to support 4096 > > > domains. Following similar logic, would it be ok to increase TasksMax to > > > 32768? That would accommodate 4096 domains with 8 tasks each. TasksMax also > > > supports the special value of "infinity", but that seems a bit aggressive to > > > me. > > > > KVM uses 1 thread per vCPU, so 8 pids is probably somewhat on the low > > side if considering high vcpu counts. I think we could go for something > > like 128k pids which allows for closer to 32 vcpus per guest > > It looks like the vcpu threads are under > /sys/fs/cgroup/pids/machine.slice/machine-qemu<name>kvm.scope/, where > pids.max is already 16k. I don't think these would be constrained by > attributes under /sys/fs/cgroup/pids/system.slice/libvirtd.service. In fact, > I don't see any additional tasks under > /sys/fs/cgroup/pids/system.slice/libvirtd.service/tasks when starting a KVM > guest. > > Since the libxl library runs in libvirtd process, any threads/processes it > creates are accounted for in > /sys/fs/cgroup/pids/system.slice/libvirtd.service. We are probably safe with > the more conservative 32k. Oh I see, I didn't realize it was a xen specific issue. I agree with your suggestion Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
