2010/7/5 Gleb Natapov <gleb@xxxxxxxxxx>:
> On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote:
>> 2010/7/5 Gleb Natapov <gleb@xxxxxxxxxx>:
>> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote:
>> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote:
>> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@xxxxxxxxxxxxxxx> wrote:
>> >> >> Hello all,
>> >> >>
>> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after
>> >> >> initializing the serial port when the drive if=virtio, but not when
>> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk"
>> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly.
>> >> >>
>> >> >> Reproduction:
>> >> >>
>> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms
>> >> >> +0ubuntu9) it will work properly:
>> >> >>
>> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \
>> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus
>> >> >>
>> >> >> As expected, the kernel panics unable to mount root (good-boot.png).
>> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes.
>> >> >>
>> >> >> ---However---if I use today's git (2010-07-01) of kvm:
>> >> >>
>> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \
>> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus
>> >> >>
>> >> >> This hangs just after initializing the Serial device (obtained by adding
>> >> >> -serial stdio -append console=ttyS0):
>> >> >>
>> >> >> Note that this only happens with the disk interface set to virtio
>> >> >> (if=virtio). It works fine for ide (if=ide).
>> >> >>
>> >> >>
>> >> >> Am I doing something wrong here?
>> >> >> Is anyone else having this problem?
>> >> >
>> >> > I have seen this issue with a RHEL 5.5 guest running under
>> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described
>> >> > with the RHEL 5.5 kernel. I have not investigated.
>> >>
>> >> This issue is affected by extboot, a feature that enables booting from
>> >> virtio-blk devices. I have just sent a patch to the KVM mailing list
>> >> to restore extboot functionality which has been broken in
>> >> qemu-kvm.git. That patch can be used to work around this issue by
>> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5
>> >> kernel hangs during serial initialization when extboot is not present.
>> >>
>> > Hang that happens during guest boot (after bootloader started the
>> > kernel) cannot be worked around by extboot. extboot is also not needed
>> > with latest qemu git to boot from virtio disks since the support for
>> > that is in the bios now.
>>
>> I agree that something else is going on here and needs to be
>> investigated, but I do think that extboot can indirectly affect the
>> guest boot.
>>
>> With extboot the virtio-blk PCI adapter is not touched by the
>> firmware/bootloader. Is it possible that a virtio-blk interrupt is
>> raised and not acknowledged before entering Linux. When Linux brings
>> up the serial port it gets swamped with interrupts? That's just a
>> guess.
>>
> That is possible when bios is actually used to boot the guest, but bug
> reporter uses -kernel option so no bios boot code should run at all.
> Virtio is initialized anyway, but this will happen with boot=on too.
Okay, I got to the bottom of this. Here's the story, see bottom of the mail for
the solution and workarounds:
It turns out that -kernel does involve the BIOS. KVM pulls apart the bzImage
and makes it available via the fw_cfg interface. The linuxboot.bin option ROM
is executed by the BIOS inside the VM to actually jump into the kernel. This
means the BIOS does POST and sets itself up; the kernel's real-mode boot
code is going to use BIOS interrupts.
So now the VM has booted, BIOS finished POST and executed linuxboot.bin,
linuxboot.bin transferred control to Linux. Then, in Linux arch/x86/boot/edd.c
a disk read to sector=0 bytes=512 is made using INT 13h. Since this disk
read comes from the Linux kernel, it happens regardless of -kernel or not.
The disk read is serviced by the BIOS. Older versions of SeaBIOS leave the
interrupt raised here, so then the kernel hangs in serial initialization later.
However, there is a simple workaround:
x86_64-softmmu/qemu-system-x86_64 -m 512 -drive
file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64
-append edd=skipmbr
When the edd=skipmbr kernel parameter is used, the kernel will not perform
the disk read and the interrupt will never get stuck. The VM boots
successfully.
The edd=skipmbr workaround is not enough when booting from disk and not
-kernel. The BIOS and bootloader will invoke INT 13h and the only way
around that is to use extboot.bin with boot=on as I originally described.
The root cause was fixed in SeaBIOS commit:
commit 4030db0d2c5a79de2a1b5c31514503e4ff2a3cd1
Author: Gleb Natapov <gleb@xxxxxxxxxx>
Date: Mon May 17 16:27:27 2010 +0300
fix two issues with virtio-blk
1. Check if blk_size is valid in virtio_blk config.
2. Disable interrupt otherwise interrupt may stuck
with some guests.
If you build SeaBIOS from source and launch KVM with -bios
path/to/seabios/out/bios.bin, RHEL5.5 will boot without hanging at serial
initialization time.
QEMU and KVM need to grab a later SeaBIOS build that contains 4030db0.
Stefan
PS: If you test this on qemu.git, you may find that the kernel doesn't hang
at serial initialization, despite the bug being present in the BIOS. I'm not
sure how QEMU gets away with it but I wonder if the interrupt configuration
is different.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
