On Mon, 14 Dec 2015 16:27:54 -0500 John Ferlan <jferlan@xxxxxxxxxx> wrote: > > > On 11/13/2015 11:56 AM, Henning Schild wrote: > > Hi, > > > > i already explained some of the cgroup problems in some detail so i > > will not do that again. > > https://www.redhat.com/archives/libvir-list/2015-October/msg00876.html > > > > I managed to solve some of the problems in the current codebase, and > > am now sharing the patches. But they are really just half of what i > > had to change to get libvirt to behave in a system with isolated > > cpus. > > > > Other changes/hacks i am not sending here because they do not work > > for the general case: > > - create machine.slice before starting libvirtd (smaller than root) > > ... and hope it wont grow > > - disabling cpuset.cpus inheritance in libvirtd > > - allowing only xml with fully specified cputune > > - set machine cpuset to (vcpupins | emulatorpin) > > > > I am not sure how useful the individual fixes are, i am sending them > > as concrete examples for the problems i described earlier. And i am > > hoping that will start a discussion. > > > > Henning > > > > Henning Schild (3): > > util: cgroups do not implicitly add task to new machine cgroup > > qemu: do not put a task into machine cgroup > > qemu cgroups: move new threads to new cgroup after cpuset is set > > up > > > > src/lxc/lxc_cgroup.c | 6 ++++++ > > src/qemu/qemu_cgroup.c | 23 ++++++++++++++--------- > > src/util/vircgroup.c | 22 ---------------------- > > 3 files changed, 20 insertions(+), 31 deletions(-) > > > > > The updated code looks fine to me - although it didn't directly git am > -3 to top of tree - I was able to make a few adjustments to get things > merged... Since no one has objected to this ordering change - I've > pushed. Sorry the patches where still based on v1.2.19. Thanks for the merge and accepting them! Wrong operation ordering within libvirt cgroups (like the ones fixed by the patches) could still push tasks onto dedicated cpus. And more importantly other cgroups users can still grab the dedicated cpus as well. The only reliable solution to prevent that seems to be making use of the "exclusive" feature of cpusets. And that would imply changing the cgroups layout of libvirt again. Because sets can not be partially exclusive and libvirt deals with dedicated cpus and shared ones. How to deal with these problems is a discussion that i wanted to get started with this patch-series. It would be nice to receive general comments on that. How should we proceed here? I could maybe write an RFC mail describing the problems again and suggesting changes to libvirt on a conceptual basis. But until then maybe people responsible for cgroups in libvirt (Paul and Martin?) can again look at https://www.redhat.com/archives/libvir-list/2015-October/msg00876.html There i described how naive use of cgoups can place tasks on cpus that are supposed to be isolated/dedicated/exclusive. Even if libvirt does not make these mistakes it should protect itself against docker, systemd, ... Henning -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list