On Thu, May 6, 2010 at 7:40 PM, Ryota Ozaki <ozaki.ryota@xxxxxxxxx> wrote:
> Through conversation with Kumar L Srikanth-B22348, I found
> that the function of getting memory usage (e.g., virsh dominfo)
> doesn't work for lxc with ns subsystem of cgroup enabled.
>
> This is because of features of ns and memory subsystems.
> Ns creates child cgroup on every process fork and as a result
> processes in a container are not assigned in a cgroup for
> domain (e.g., libvirt/lxc/test1/). For example, libvirt_lxc
> and init (or somewhat specified in XML) are assigned into
> libvirt/lxc/test1/8839/ and libvirt/lxc/test1/8839/8849/,
> respectively. On the other hand, memory subsystem accounts
> memory usage within a group of processes by default, i.e.,
> it does not take any child (and descendent) groups into
> account. With the two features, virsh dominfo which just
> checks memory usage of a cgroup for domain always returns
> zero because the cgroup has no process.
>
> Setting memory.use_hierarchy of a group allows to account
> (and limit) memory usage of every descendent groups of the group.
> By setting it of a cgroup for domain, we can get proper memory
> usage of lxc with ns subsystem enabled. (To be exact, the
> setting is required only when memory and ns subsystems are
> enabled at the same time, e.g., mount -t cgroup none /cgroup.)
> ---
This does sound like a valid use case and the correct fix.
> src/util/cgroup.c | 49 +++++++++++++++++++++++++++++++++++++++++++++----
> 1 files changed, 45 insertions(+), 4 deletions(-)
>
> diff --git a/src/util/cgroup.c b/src/util/cgroup.c
> index b8b2eb5..93cd6a9 100644
> --- a/src/util/cgroup.c
> +++ b/src/util/cgroup.c
> @@ -443,7 +443,38 @@ static int virCgroupCpuSetInherit(virCgroupPtr parent, virCgroupPtr group)
> return rc;
> }
>
> -static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, int create)
> +static int virCgroupSetMemoryUseHierarchy(virCgroupPtr group)
> +{
> + int rc = 0;
> + unsigned long long value;
> + const char *filename = "memory.use_hierarchy";
> +
> + rc = virCgroupGetValueU64(group,
> + VIR_CGROUP_CONTROLLER_MEMORY,
> + filename, &value);
> + if (rc != 0) {
> + VIR_ERROR("Failed to read %s/%s (%d)", group->path, filename, rc);
> + return rc;
> + }
> +
> + /* Setting twice causes error, so if already enabled, skip setting */
> + if (value == 1)
> + return 0;
> +
> + VIR_DEBUG("Setting up %s/%s", group->path, filename);
> + rc = virCgroupSetValueU64(group,
> + VIR_CGROUP_CONTROLLER_MEMORY,
> + filename, 1);
> +
> + if (rc != 0) {
> + VIR_ERROR("Failed to set %s/%s (%d)", group->path, filename, rc);
> + }
> +
> + return rc;
> +}
> +
> +static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group,
> + int create, int memory_hierarchy)
> {
> int i;
> int rc = 0;
> @@ -477,6 +508,16 @@ static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, int creat
> break;
> }
> }
Can you please add a comment here stating that memory.use_hierarchy
should always be called prior to creating subcgroups and attaching
tasks
> + if (memory_hierarchy &&
> + group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].mountPoint != NULL &&
> + (i == VIR_CGROUP_CONTROLLER_MEMORY ||
> + STREQ(group->controllers[i].mountPoint, group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].mountPoint))) {
> + rc = virCgroupSetMemoryUseHierarchy(group);
> + if (rc != 0) {
> + VIR_FREE(path);
> + break;
> + }
> + }
> }
>
> VIR_FREE(path);
> @@ -553,7 +594,7 @@ static int virCgroupAppRoot(int privileged,
> if (rc != 0)
> goto cleanup;
>
> - rc = virCgroupMakeGroup(rootgrp, *group, create);
> + rc = virCgroupMakeGroup(rootgrp, *group, create, 0);
>
> cleanup:
> virCgroupFree(&rootgrp);
> @@ -653,7 +694,7 @@ int virCgroupForDriver(const char *name,
> VIR_FREE(path);
>
> if (rc == 0) {
> - rc = virCgroupMakeGroup(rootgrp, *group, create);
> + rc = virCgroupMakeGroup(rootgrp, *group, create, 0);
> if (rc != 0)
> virCgroupFree(group);
> }
> @@ -703,7 +744,7 @@ int virCgroupForDomain(virCgroupPtr driver,
> VIR_FREE(path);
>
> if (rc == 0) {
> - rc = virCgroupMakeGroup(driver, *group, create);
> + rc = virCgroupMakeGroup(driver, *group, create, 1);
> if (rc != 0)
> virCgroupFree(group);
> }
A comment on why Domains get hierarchy support and Drivers don't will
help unless it is very obvious to developers.
Balbir
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list