On Wed, Jul 07, 2010 at 10:03:31AM +0100, Stefan Hajnoczi wrote: > 2010/7/5 Gleb Natapov <gleb@xxxxxxxxxx>: > > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: > >> 2010/7/5 Gleb Natapov <gleb@xxxxxxxxxx>: > >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: > >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote: > >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@xxxxxxxxxxxxxxx> wrote: > >> >> >> Hello all, > >> >> >> > >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after > >> >> >> initializing the serial port when the drive if=virtio, but not when > >> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" > >> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. > >> >> >> > >> >> >> Reproduction: > >> >> >> > >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms > >> >> >> +0ubuntu9) it will work properly: > >> >> >> > >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> > >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). > >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. > >> >> >> > >> >> >> ---However---if I use today's git (2010-07-01) of kvm: > >> >> >> > >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> > >> >> >> This hangs just after initializing the Serial device (obtained by adding > >> >> >> -serial stdio -append console=ttyS0): > >> >> >> > >> >> >> Note that this only happens with the disk interface set to virtio > >> >> >> (if=virtio). It works fine for ide (if=ide). > >> >> >> > >> >> >> > >> >> >> Am I doing something wrong here? > >> >> >> Is anyone else having this problem? > >> >> > > >> >> > I have seen this issue with a RHEL 5.5 guest running under > >> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described > >> >> > with the RHEL 5.5 kernel. I have not investigated. > >> >> > >> >> This issue is affected by extboot, a feature that enables booting from > >> >> virtio-blk devices. I have just sent a patch to the KVM mailing list > >> >> to restore extboot functionality which has been broken in > >> >> qemu-kvm.git. That patch can be used to work around this issue by > >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 > >> >> kernel hangs during serial initialization when extboot is not present. > >> >> > >> > Hang that happens during guest boot (after bootloader started the > >> > kernel) cannot be worked around by extboot. extboot is also not needed > >> > with latest qemu git to boot from virtio disks since the support for > >> > that is in the bios now. > >> > >> I agree that something else is going on here and needs to be > >> investigated, but I do think that extboot can indirectly affect the > >> guest boot. > >> > >> With extboot the virtio-blk PCI adapter is not touched by the > >> firmware/bootloader. Is it possible that a virtio-blk interrupt is > >> raised and not acknowledged before entering Linux. When Linux brings > >> up the serial port it gets swamped with interrupts? That's just a > >> guess. > >> > > That is possible when bios is actually used to boot the guest, but bug > > reporter uses -kernel option so no bios boot code should run at all. > > Virtio is initialized anyway, but this will happen with boot=on too. > > Okay, I got to the bottom of this. Here's the story, see bottom of the mail for > the solution and workarounds: > Great. Thanks Stefan. I was sure that fix you found is in qemu.git already. It is good to know that Linux actually uses int13 once. Didn't know that. To be absolutely sure no interrupt will stuck we should probably clear interrupt after each virtio read in bios by reading status register. > It turns out that -kernel does involve the BIOS. KVM pulls apart the bzImage > and makes it available via the fw_cfg interface. The linuxboot.bin option ROM > is executed by the BIOS inside the VM to actually jump into the kernel. This > means the BIOS does POST and sets itself up; the kernel's real-mode boot > code is going to use BIOS interrupts. > > So now the VM has booted, BIOS finished POST and executed linuxboot.bin, > linuxboot.bin transferred control to Linux. Then, in Linux arch/x86/boot/edd.c > a disk read to sector=0 bytes=512 is made using INT 13h. Since this disk > read comes from the Linux kernel, it happens regardless of -kernel or not. > > The disk read is serviced by the BIOS. Older versions of SeaBIOS leave the > interrupt raised here, so then the kernel hangs in serial initialization later. > > However, there is a simple workaround: > > x86_64-softmmu/qemu-system-x86_64 -m 512 -drive > file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64 > -append edd=skipmbr > > When the edd=skipmbr kernel parameter is used, the kernel will not perform > the disk read and the interrupt will never get stuck. The VM boots > successfully. > > The edd=skipmbr workaround is not enough when booting from disk and not > -kernel. The BIOS and bootloader will invoke INT 13h and the only way > around that is to use extboot.bin with boot=on as I originally described. > > The root cause was fixed in SeaBIOS commit: > > commit 4030db0d2c5a79de2a1b5c31514503e4ff2a3cd1 > Author: Gleb Natapov <gleb@xxxxxxxxxx> > Date: Mon May 17 16:27:27 2010 +0300 > > fix two issues with virtio-blk > > 1. Check if blk_size is valid in virtio_blk config. > 2. Disable interrupt otherwise interrupt may stuck > with some guests. > > If you build SeaBIOS from source and launch KVM with -bios > path/to/seabios/out/bios.bin, RHEL5.5 will boot without hanging at serial > initialization time. > > QEMU and KVM need to grab a later SeaBIOS build that contains 4030db0. > > Stefan > > PS: If you test this on qemu.git, you may find that the kernel doesn't hang > at serial initialization, despite the bug being present in the BIOS. I'm not > sure how QEMU gets away with it but I wonder if the interrupt configuration > is different. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
