Re: Failed to access console after launching LXC through libvirt

unicell <unicell@xxxxxxxxx> · Fri, 9 Nov 2012 00:30:23 +0800

On Fri, Nov 9, 2012 at 12:09 AM, unicell <unicell@xxxxxxxxx> wrote:

Hi,
I'm trying to manage LXC instances through OpenStack, which use libvirt as a virtualization driver layer. After launching LXC instance, I simply could not attach to the console.

virsh # list
 Id Name                 State
----------------------------------
14366 instance-00000078    running

virsh # console 14366

Connected to domain instance-00000078
Escape character is ^]

And it keeps stuck here.

Initially I thought it could be a cgroup device config issue, and tried to "echo ‘c 5:1 rwm' >/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow“, to add /dev/console access right to the container. But it does not work, console stilll not working.

Before the cgroup tweaking, devices.list are as follows

[root@localhost libvirt]# cat /cgroup/devices/libvirt/lxc/instance-0000007a/devices.list

c 1:3 rwm
c 1:5 rwm
c 1:7 rwm
c 1:8 rwm
c 1:9 rwm
c 5:0 rwm
c 5:2 rwm

c 136:* rwm

I'm not quite sure what kind of issue could possibly cause this console access issue. Cause I'm now using two different guest OS templates on two different host OSes. Following are some symptoms I observed.

* (g) for container guest OS
* (h) for host OS
* NOTE: Ubuntu guest os are generated by lxc-create -t ubuntu, which is an Ubuntu 12.04 based lxc template.

             CentOS guest os are handcrafted by myself, taking reference of http://wiki.1tux.org/wiki/Lxc/Installation/Guest/Centos/6

Ubuntu (g) launched by lxc-start on Ubuntu 12.04 host, console works
Ubuntu (g) launched by OpenStack / libvirt on Ubuntu 12.04 host, console works
Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host, console NOT works

CentOS (g) launched by lxc-start on Ubuntu 12.04 host, console works
CentOS (g) launched by OpenStack / libvirt on Ubuntu12.04 host, console NOT works
CentOS (g) launched by OpenStack / libvirt on CentOS 6.3 host, console NOT works

So somehow I feel it could be something related to LXC template rootfs, but Host OS do make a difference (2 vs. 3). lxc-start / libvirt also makes a difference (4 vs. 5), but that might be caused by cgroup setting.

Could someone shed me some clues to further digging this issue? Thanks!  

--
Qiu Yu

For the "Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host", I also tried following command and console still not working. 

echo "a *.* rwm" >/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow

More information attached.

[root@localhost ~]# ls -l /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty*

crw-rw-rw- 1 root tty 5, 0 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty
crw------- 1 root tty 4, 0 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty0

crw------- 1 root tty 4, 1 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty1
crw------- 1 root tty 4, 2 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty2

crw------- 1 root tty 4, 3 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty3
crw------- 1 root tty 4, 4 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty4

crw------- 1 root tty 4, 5 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty5
crw------- 1 root tty 4, 6 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty6

crw------- 1 root tty 4, 7 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty7
crw------- 1 root tty 4, 8 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty8

crw------- 1 root tty 4, 9 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty9

[root@localhost ~]# cat /cgroup/devices/libvirt/lxc/instance-00000078/devices.list 

c 1:3 rwm
c 1:5 rwm
c 1:7 rwm
c 1:8 rwm
c 1:9 rwm
c 5:0 rwm
c 5:2 rwm
c 136:* rwm
c 5:1 rwm
c 5:* rwm
a *:* rwm

[root@localhost ~]# cat /etc/libvirt/lxc/instance-00000078.xml 
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE

OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh edit instance-00000078
or other application using the libvirt API.
-->

<domain type='lxc'>

  <name>instance-00000078</name>
  <uuid>55f225d1-e49c-4a51-9c17-0a5dde6d1917</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>

  <vcpu>2</vcpu>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/sbin/init</init>
    <cmdline>console=ttyS0</cmdline>

  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>

  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/home/stack/nova_state/instances/instance-00000078/rootfs'/>

      <target dir='/'/>
    </filesystem>
    <interface type='bridge'>
      <mac address='fa:16:3e:08:4f:b9'/>
      <source bridge='br100'/>

      <filterref filter='nova-instance-instance-00000078-fa163e084fb9'>
        <parameter name='DHCPSERVER' value='10.48.253.1'/>
        <parameter name='IP' value='10.48.253.2'/>

        <parameter name='PROJMASK' value='255.255.255.0'/>
        <parameter name='PROJNET' value='10.48.253.0'/>
      </filterref>
    </interface>

    <console type='pty'>
      <target type='lxc' port='0'/>
    </console>
  </devices>
</domain>

--
Qiu Yu

_______________________________________________
libvirt-users mailing list
libvirt-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvirt-users