2010/7/7 Gleb Natapov <gleb@xxxxxxxxxx>: > On Wed, Jul 07, 2010 at 10:03:31AM +0100, Stefan Hajnoczi wrote: >> 2010/7/5 Gleb Natapov <gleb@xxxxxxxxxx>: >> > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: >> >> 2010/7/5 Gleb Natapov <gleb@xxxxxxxxxx>: >> >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: >> >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote: >> >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@xxxxxxxxxxxxxxx> wrote: >> >> >> >> Hello all, >> >> >> >> >> >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after >> >> >> >> initializing the serial port when the drive if=virtio, but not when >> >> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" >> >> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. >> >> >> >> >> >> >> >> Reproduction: >> >> >> >> >> >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms >> >> >> >> +0ubuntu9) it will work properly: >> >> >> >> >> >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> >> >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). >> >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. >> >> >> >> >> >> >> >> ---However---if I use today's git (2010-07-01) of kvm: >> >> >> >> >> >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> >> >> >> >> This hangs just after initializing the Serial device (obtained by adding >> >> >> >> -serial stdio -append console=ttyS0): >> >> >> >> >> >> >> >> Note that this only happens with the disk interface set to virtio >> >> >> >> (if=virtio). It works fine for ide (if=ide). >> >> >> >> >> >> >> >> >> >> >> >> Am I doing something wrong here? >> >> >> >> Is anyone else having this problem? >> >> >> > >> >> >> > I have seen this issue with a RHEL 5.5 guest running under >> >> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described >> >> >> > with the RHEL 5.5 kernel. I have not investigated. >> >> >> >> >> >> This issue is affected by extboot, a feature that enables booting from >> >> >> virtio-blk devices. I have just sent a patch to the KVM mailing list >> >> >> to restore extboot functionality which has been broken in >> >> >> qemu-kvm.git. That patch can be used to work around this issue by >> >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 >> >> >> kernel hangs during serial initialization when extboot is not present. >> >> >> >> >> > Hang that happens during guest boot (after bootloader started the >> >> > kernel) cannot be worked around by extboot. extboot is also not needed >> >> > with latest qemu git to boot from virtio disks since the support for >> >> > that is in the bios now. >> >> >> >> I agree that something else is going on here and needs to be >> >> investigated, but I do think that extboot can indirectly affect the >> >> guest boot. >> >> >> >> With extboot the virtio-blk PCI adapter is not touched by the >> >> firmware/bootloader. Is it possible that a virtio-blk interrupt is >> >> raised and not acknowledged before entering Linux. When Linux brings >> >> up the serial port it gets swamped with interrupts? That's just a >> >> guess. >> >> >> > That is possible when bios is actually used to boot the guest, but bug >> > reporter uses -kernel option so no bios boot code should run at all. >> > Virtio is initialized anyway, but this will happen with boot=on too. >> >> Okay, I got to the bottom of this. Here's the story, see bottom of the mail for >> the solution and workarounds: >> > Great. Thanks Stefan. I was sure that fix you found is in qemu.git > already. It is good to know that Linux actually uses int13 once. Didn't > know that. To be absolutely sure no interrupt will stuck we should probably > clear interrupt after each virtio read in bios by reading status register. Yes, this is the approach I took when rewriting gPXE's virtio-net driver recently: http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=blob;f=src/drivers/net/virtio-net.c;h=50c580049c3663f0d1332d51e30c8a59b5663b68;hb=d58bf6fc8db4a55075e370c7e1d436a570400f55#l299 Are you spinning a patch or should I submit one? >> It turns out that -kernel does involve the BIOS. KVM pulls apart the bzImage >> and makes it available via the fw_cfg interface. The linuxboot.bin option ROM >> is executed by the BIOS inside the VM to actually jump into the kernel. This >> means the BIOS does POST and sets itself up; the kernel's real-mode boot >> code is going to use BIOS interrupts. >> >> So now the VM has booted, BIOS finished POST and executed linuxboot.bin, >> linuxboot.bin transferred control to Linux. Then, in Linux arch/x86/boot/edd.c >> a disk read to sector=0 bytes=512 is made using INT 13h. Since this disk >> read comes from the Linux kernel, it happens regardless of -kernel or not. >> >> The disk read is serviced by the BIOS. Older versions of SeaBIOS leave the >> interrupt raised here, so then the kernel hangs in serial initialization later. >> >> However, there is a simple workaround: >> >> x86_64-softmmu/qemu-system-x86_64 -m 512 -drive >> file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64 >> -append edd=skipmbr >> >> When the edd=skipmbr kernel parameter is used, the kernel will not perform >> the disk read and the interrupt will never get stuck. The VM boots >> successfully. >> >> The edd=skipmbr workaround is not enough when booting from disk and not >> -kernel. The BIOS and bootloader will invoke INT 13h and the only way >> around that is to use extboot.bin with boot=on as I originally described. >> >> The root cause was fixed in SeaBIOS commit: >> >> commit 4030db0d2c5a79de2a1b5c31514503e4ff2a3cd1 >> Author: Gleb Natapov <gleb@xxxxxxxxxx> >> Date: Mon May 17 16:27:27 2010 +0300 >> >> fix two issues with virtio-blk >> >> 1. Check if blk_size is valid in virtio_blk config. >> 2. Disable interrupt otherwise interrupt may stuck >> with some guests. >> >> If you build SeaBIOS from source and launch KVM with -bios >> path/to/seabios/out/bios.bin, RHEL5.5 will boot without hanging at serial >> initialization time. >> >> QEMU and KVM need to grab a later SeaBIOS build that contains 4030db0. >> >> Stefan >> >> PS: If you test this on qemu.git, you may find that the kernel doesn't hang >> at serial initialization, despite the bug being present in the BIOS. I'm not >> sure how QEMU gets away with it but I wonder if the interrupt configuration >> is different. > > -- > Gleb. > Stefan -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
