wow! i was starting to think i would never get past this problem! [root@PHS-001 ~]# echo $VIRSH_DEFAULT_CONNECT_URI lxc:/// [root@PHS-001 ~]# virsh create /vps/def/exec/sys/arch-nano.xml Domain arch-nano created from /vps/def/exec/sys/arch-nano.xml [root@PHS-001 ~]# virsh console arch-nano Connected to domain arch-nano Escape character is ^] # [root@PHS-001 ~]# grep cgroup /proc/mounts none /cgroup cgroup rw,relatime,devices,memory,cpuacct,cpu 0 0 everything seems to be working fine now after disabling "ns" from cgroup mount; thank you very much Ryota and Daniel for you time and assistance. i must have sunk 40+ hours into this, but i learned a tremendous amount, so nothing is lost. :-) this is somewhat of a digression, and i can submit a new thread if need be, but my last remaining question is about the <filesystem> tag/root mounting. i have not found any mention of the <filesystem> tag in the docs or how to work with it... i want to have libvirt mount a specified device (a btrfs subvolume) and use it for the root filesystem of the created LXC container. is there a way to do this? perhaps <source dev="/dev/sdb" options="subvol....."/>, or do i need to use the storage XML? i didnt see a way to use a storage/device definition as the rootfs. im eager to get working with python bindings, just trying to see it all work with XML first. thanks++ On Thu, Dec 10, 2009 at 6:45 AM, Ryota Ozaki <ozaki.ryota@xxxxxxxxx> wrote: > On Thu, Dec 10, 2009 at 9:36 PM, Daniel P. Berrange <berrange@xxxxxxxxxx> wrote: >> On Thu, Dec 10, 2009 at 09:26:39PM +0900, Ryota Ozaki wrote: >>> On Thu, Dec 10, 2009 at 9:03 PM, Daniel P. Berrange <berrange@xxxxxxxxxx> wrote: >>> > On Thu, Dec 10, 2009 at 02:22:37AM -0600, Tony Risinger wrote: >>> >> i'm trying to get even the simplest busybox container with libvirt+LXC >>> >> with very limited success. I feel l am missing something supremely >>> >> simple for me to be hung on this for weeks. >>> >> >>> >> i dont see anything interesting in domain log, but getting this error >>> >> from "LIBVIRT_DEBUG=1 libvirtd": >>> >> >>> >> 05:27:56.113: error : lxcDomainGetInfo:462 : internal error Unable to >>> >> get cgroup for arch-nano >>> >> 05:27:56.113: debug : virDomainFree:2004 : domain=0x81d8e68 >>> >> 05:27:56.113: debug : virUnrefDomain:422 : unref domain 0x81d8e68 arch-nano 1 >>> >> 05:27:56.113: debug : virReleaseDomain:376 : release domain 0x81d8e68 arch-nano >>> >> 05:27:56.113: debug : virReleaseDomain:392 : unref connection 0x81dc0f0 2 >>> >> 05:27:56.113: debug : remoteSerializeError:141 : prog=536903814 ver=1 >>> >> proc=16 type=1 serial=4, msg=internal error Unable to get cgroup for >>> >> arch-nano >>> >> >>> >> i've been using this root filesystem layout: >>> >> >>> >> [root@PHS-001 arch-nano]# tree >>> >> . >>> >> |-- bin >>> >> | |-- cat -> ../sbin/busybox >>> >> | |-- chdir -> ../sbin/busybox >>> >> | |-- chmod -> ../sbin/busybox >>> >> | |-- ls -> ../sbin/busybox >>> >> | |-- rm -> ../sbin/busybox >>> >> | |-- sh -> ../sbin/busybox >>> >> | `-- vi -> ../sbin/busybox >>> >> |-- dev >>> >> | `-- pts >>> >> |-- etc >>> >> |-- proc >>> >> |-- sbin >>> >> | |-- busybox >>> >> | `-- init -> busybox >>> >> `-- sys >>> >> >>> >> all folders besides /bin and /sbin were created by libvirt. i tried >>> >> using the /sbin/init script previously suggested: >>> >> >>> >> #!/sbin/busybox >>> >> sh >>> > >>> > >>> > Sorry, my suggestion was wrong. I forgot that if you have #!/sbin/busybox >>> > it will attempt to execute the command matching the name of the script. >>> > So it will in fact try to run 'init', rather than 'sh'. >>> > >>> > Just make the libvirt XML point directly to /bin/sh instead and it >>> > should work. I even tested it this time :-) >>> >>> Hem, I still have a problem with ns subsystem enabled. Yes, I can launch >>> a container however the cgroup hierarchy is wrong from libvirtd expecting >>> like: >>> >>> /: libvirtd --daemon >>> /5345: /usr/libexec/libvirt_lxc --name >>> >>> Daniel, could you confirm how about your cgroup hierarchy? >> >> What you do mean by 'ns' subsystem ? > > 'ns' is one of functions of cgroups like such as devices, memory, > cpu, etc. and it is enabled if you mount cgroup without any options > that Tony is doing. > >> >> >> # grep cgroup /proc/mounts >> cgroup /dev/cgroups/cpu cgroup rw,relatime,cpuacct,cpu 0 0 >> cgroup /dev/cgroups/memory cgroup rw,relatime,memory 0 0 >> cgroup /dev/cgroups/devices cgroup rw,relatime,devices 0 0 > > Oh, you don't enable 'ns', so yes, things go fine in your environment. > > >> >> # cat /proc/`pgrep libvirtd`/cgroup >> 32:devices:/sysdefault >> 16:memory:/sysdefault >> 12:cpuacct,cpu:/sysdefault >> >> # cat /proc/`pgrep libvirt_lxc`/cgroup >> 32:devices:/sysdefault/libvirt/lxc/vm1 >> 16:memory:/sysdefault/libvirt/lxc/vm1 >> 12:cpuacct,cpu:/sysdefault/libvirt/lxc/vm1 >> >> >> And the process inside the contanier is PID 12309 >> >> # cat /proc/12309/cgroup >> 32:devices:/sysdefault/libvirt/lxc/vm1 >> 16:memory:/sysdefault/libvirt/lxc/vm1 >> 12:cpuacct,cpu:/sysdefault/libvirt/lxc/vm1 >> >> >> Which all appears to be correct to me >> >> This is on a Fedora 12 host 2.6.31.6-145.fc12.i686.PAE with >> >> CONFIG_UTS_NS=y >> CONFIG_IPC_NS=y >> CONFIG_USER_NS=y >> CONFIG_PID_NS=y >> CONFIG_NET_NS=y >> CONFIG_CGROUP_SCHED=y >> CONFIG_CGROUPS=y >> # CONFIG_CGROUP_DEBUG is not set >> CONFIG_CGROUP_NS=y > > This is the function I'm mentioning. > > ozaki-r > >> CONFIG_CGROUP_FREEZER=y >> CONFIG_CGROUP_DEVICE=y >> CONFIG_CGROUP_CPUACCT=y >> CONFIG_CGROUP_MEM_RES_CTLR=y >> CONFIG_CGROUP_MEM_RES_CTLR_SWAP=y >> CONFIG_NET_CLS_CGROUP=y >> >> >> Regards, >> Daniel >> -- >> |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| >> |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| >> |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| >> |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| >> > -- Libvir-list mailing list Libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list