On 04/11/2012 03:44 PM, Guido Winkelmann wrote: > Hi, > > Nested virtualization on Intel does not work for me with qemu-kvm. As soon as > the third layer OS (second virtualised) is starting the Linux kernel, the > entire second layer freezes up. The last thing I can see console of the third > layer system before it freezes is "Decompressing Linux... ". (no "done", > though). When starting without nofb option, the kernel still manages to set > the screen resolution before freezing. > > Grub/Syslinux still work, but are extremely slow. > > Both the first layer OS (i.e. the one running on bare metal) and the second > layer OS are 64-bit-Fedora 16 with Kernel 3.3.1-3.fc16.x86_64. On both the > first and second layer OS, the kvm_intel modules are loaded with nested=Y > parameter. (I've also tried with nested=N in the second layer. Didn't change > anything.) > Qemu-kvm was originally the Fedora-shipped 0.14, but I have since upgraded to > 1.0. (Using rpmbuild with the specfile and patches from > http://pkgs.fedoraproject.org/gitweb/?p=qemu.git;a=blob;f=qemu.spec;hb=HEAD) > > The second layer machine has this CPU specification in libvirt on the first > layer OS: > > <cpu mode='custom' match='exact'> > <model fallback='allow'>Nehalem</model> > <feature policy='require' name='vmx'/> > </cpu> > > which results in this qemu commandline (from libvirt's logs): > > LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu- > kvm -S -M pc-0.15 -cpu kvm64,+lahf_lm,+popcnt,+sse4.2,+sse4.1,+ssse3,+vmx - > enable-kvm -m 8192 -smp 8,sockets=8,cores=1,threads=1 -name vshost1 -uuid > 192b8c4b-0ded-07aa-2545-d7fef4cd897f -nodefconfig -nodefaults -chardev > socket,id=charmonitor,path=/var/lib/libvirt/qemu/vshost1.monitor,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown - > no-acpi -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > file=/data/vshost1.img,if=none,id=drive-virtio-disk0,format=qcow2 -device > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio- > disk0,bootindex=1 -drive file=/data/Fedora-16-x86_64- > netinst.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw - > device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > tap,fd=21,id=hostnet0,vhost=on,vhostfd=22 -device virtio-net- > pci,netdev=hostnet0,id=net0,mac=52:54:00:84:7d:46,bus=pci.0,addr=0x3 -netdev > tap,fd=23,id=hostnet1,vhost=on,vhostfd=24 -device virtio-net- > pci,netdev=hostnet1,id=net1,mac=52:54:00:84:8d:46,bus=pci.0,addr=0x4 -vnc > 127.0.0.1:0,password -k de -vga cirrus -device virtio-balloon- > pci,id=balloon0,bus=pci.0,addr=0x6 > > I have also tried some other combinations for the cpu element, like changing > the model to core2duo and/or including all the features reported by libvirt's > capabalities command. > > The third level machine does not have a cpu element in libvirt, and its > commandline looks like this: > > LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu- > kvm -S -M pc-0.14 -enable-kvm -m 8192 -smp 4,sockets=4,cores=1,threads=1 -name > gentoo -uuid 3cdcc902-4520-df25-92ac-31ca5c707a50 -nodefconfig -nodefaults - > chardev > socket,id=charmonitor,path=/var/lib/libvirt/qemu/gentoo.monitor,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-acpi -drive > file=/data/gentoo.img,if=none,id=drive-virtio-disk0,format=qcow2 -device > virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 - > drive file=/data/install-amd64- > minimal-20120223.iso,if=none,media=cdrom,id=drive- > ide0-1-0,readonly=on,format=raw -device ide- > drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev > tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net- > pci,netdev=hostnet0,id=net0,mac=52:54:00:84:6d:46,bus=pci.0,addr=0x3 -usb -vnc > 127.0.0.1:0,password -k de -vga cirrus -device virtio-balloon- > pci,id=balloon0,bus=pci.0,addr=0x5 > > The third layer OS is a recent Gentoo minimal install (amd64), but somehow I > don't think that matters at this point... > > The metal is a Dell PowerEdge R710 server with two Xeon E5520 CPUs. I've tried > updating the machine's BIOS and other firmware to the latest version. That > took a lot of time and a lot of searching on Dell websites, but didn't change > anything. > > Does anyone have any idea what might be going wrong here or how I could debug > this further? I'm not sure if this is the problem but I noticed that the second layer and the third layer have the same memory size (8G), how about trying to reduce the memory for the third layer ? Orit > > Guido > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
