On Wed, 2017-02-15 at 09:24 +0000, Daniel P. Berrange wrote: [...] > $ virsh start f22-arm32 > error: Failed to start domain f22-arm32 > error: internal error: qemu unexpectedly closed the monitor: 2017-02-15T09:24:03.967648Z qemu-system-arm: -device ioh3420,port=0x8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1: MSI is not supported by interrupt controller > 2017-02-15T09:24:03.968154Z qemu-system-arm: -device ioh3420,port=0x8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1: Device initialization failed > > I've not bisected it other than to find it works in 2.5.0 and is > broken in 3.0.0 I'd like you to share a few additional details: * What does the guest XML look like after libvirt has had a change to augment it? * What QEMU command line does libvirt generate based on said guest XML? * What QEMU version are you running? I'm asking because I've just spent some time trying to run ARM guests[1] on my laptop[2] and I can't reproduce the failure you're describing, so I need more information to try and narrow it down. After the skip, a bunch of unstructured rambling about stuff I've discovered in the process, interleaved with more questions about your issue. Have fun! :) --- I would expect such an error to pop up simply by running $ qemu-system-arm -nodefaults -M virt \ -device ioh3420,port=0x8,... Does that happen on your system? --- I didn't realize the mach-virt machine type was not an aarch64-only thing. Indeed, it's available through qemu-system-arm too, and the hardware seems to be the same, eg. running $ echo -e 'info qtree\nquit\n' | \ qemu-system-$arch -M virt -monitor stdio yields the same output. The only difference AFAICT is that qemu-system-arm limits the selection of CPUs to 32-bit models only, eg. cortex-a15 is available on both but only qemu-system-aarch64 lets you use cortex-a57. --- I know 32-bit UEFI is a thing, because it's used on a bunch of budget x86 tablet and causes grief and pain to anyone trying to run Linux on them. However, Fedora only ships 64-bit binaries (edk2-ovmf and edk2-aarch64 packages) so I can't really try whether an armv7l guest can boot using UEFI. --- Speaking of booting the guest, how would that work with the guest XML you're feeding libvirt, exactly? Since you don't have any sort of firmware, the only way I can see it working is to to have <kernel>, <initrd> and <cmdline> elements inside <domain><os>, and libvirt can't possibly figure out their values for you... --- Does it help at all to use <address type='virtio-mmio'/> in order to force the the SCSI controller and network interface to use virtio-mmio instead of virtio-pci? --- As a reference, here's the XML for my Fedora 25/armv7l guest: <domain type='qemu'> <name>f25-armv7l</name> <uuid>029956e6-5e98-4ca6-8b2b-72322f73f3ea</uuid> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>1</vcpu> <os> <type arch='armv7l' machine='virt-2.9'>hvm</type> <kernel>/home/abologna/.local/share/libvirt/images/f25-armv7l.vmlinuz</kernel> <initrd>/home/abologna/.local/share/libvirt/images/f25-armv7l.initramfs</initrd> <cmdline>console=ttyAMA0 rw root=LABEL=_/ rootwait</cmdline> <boot dev='hd'/> </os> <features> <gic version='2'/> </features> <cpu mode='custom' match='exact'> <model fallback='allow'>cortex-a15</model> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/bin/qemu-system-arm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/home/abologna/.local/share/libvirt/images/f25-armv7l.qcow2'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='scsi' index='0' model='virtio-scsi'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='ioh3420'/> <target chassis='1' port='0x8'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='ioh3420'/> <target chassis='2' port='0x9'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='ioh3420'/> <target chassis='3' port='0xa'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <interface type='user'> <mac address='52:54:00:9a:47:50'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> </devices> </domain> --- And here's that for my Fedora 25/aarch64 guest: <domain type='qemu'> <name>f25-aarch64</name> <uuid>e3e954ba-5280-4b6b-9bfc-7f944f2c67b2</uuid> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>1</vcpu> <os> <type arch='aarch64' machine='virt-2.9'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader> <nvram>/home/abologna/.config/libvirt/qemu/nvram/f25-aarch64_VARS.fd</nvram> <boot dev='hd'/> </os> <features> <gic version='2'/> </features> <cpu mode='custom' match='exact'> <model fallback='allow'>cortex-a57</model> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/bin/qemu-system-aarch64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/home/abologna/.local/share/libvirt/images/f25-aarch64.qcow2'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='scsi' index='0' model='virtio-scsi'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='ioh3420'/> <target chassis='1' port='0x8'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='ioh3420'/> <target chassis='2' port='0x9'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='ioh3420'/> <target chassis='3' port='0xa'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <interface type='user'> <mac address='52:54:00:c3:33:4a'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> </devices> </domain> [1] Fedora 25/armv7l, plus Fedora 25/aarch64 for comparison [2] Fedora 24/x86_64 with libvirt 3.0.0; I've tried, in turn, QEMU master, QEMU 2.6 (as shipped with Fedora) and upstream QEMU 2.5 -- Andrea Bolognani / Red Hat / Virtualization -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
