On 11/15/2013 03:35 PM, Thomas Kuther wrote: > Hello, > > I'm trying to migrate a working qemu command line configuration to > libvirt. > The part I'm currently failing on is: > > $ qemu-system-x86_64 -M Q35 ... -device vfio-pci,host=05:00.0,bus=pcie.0 > > The right way to translate this into libvirt XML seems to be using > <hostdev>, but I seem to be unable to plug it into the pcie-root port > > This is how the interesting part looks like when I let "virsh edit" > generate an <address> > > <controller type='pci' index='0' model='pcie-root'/> > <controller type='pci' index='1' model='dmi-to-pci-bridge'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x0'/> > </controller> > <controller type='pci' index='2' model='pci-bridge'> > <address type='pci' domain='0x0000' bus='0x01' slot='0x01' > function='0x0'/> > </controller> > [...] > <hostdev mode='subsystem' type='pci' managed='yes'> > <driver name='vfio'/> > <source> > <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> > </source> > <address type='pci' domain='0x0000' bus='0x02' slot='0x06' > function='0x0'/> > </hostdev> > [...] > > To my understanding, this will plug the host device into the > pci-bridge controller. > The guest OS doesn't boot with this and resets right after bios. Ugh. That's very unfortunate. This is the first report I've heard of something failing in such a bad way due to being plugged into a pci-bridge slot; up until now I'd only heard that there is some extra PCIe functionality that would be missing if a device was plugged into a PCI slot vs. PCIe. Can I ask what type of device this is? > > Manually setting > <address type='pci' domain='0x0000' bus='0x00' slot='0x1E' > function='0x0'/> > cause XML validation failure. > > Is there any way in libvirt XML to plug a host's PCI-E device directly > into the pcie-root port, like it works on qemu command line? I'm sorry to say, no. With very few (and specific) exceptions, libvirt insists that all guest devices be plugged into a hot-pluggable PCI slot - this eliminates both the PCIe "root complex" (a.k.a. pcie.0) as well as the dmi-to-pci-controller that is plugged into pcie.0 (because pci-to-dmi controllers' slots don't support hot-plug). This is done because, for now, almost all devices that qemu knows about are PCI (no PCI-e) devices, and if we allowed plugging them into pcie.0 now, then on the day in the future when qemu begins enforcing the difference between PCI and PCIe (currently it doesn't), the world would be full of libvirt configs that would no longer work. There was some discussion about this a month or two ago either on libvir-list or maybe it was the qemu-devel list. We decided that qemu needs to provide some sort of introspection of the devices' connection types so that libvirt can determine what device can plug into which slots; at that time we'll be able to allow exactly what's proper in each case. In the meantime we're stuck with being overly cautious in order to prevent future catastrophe. > > I'm aware I could use something like > > <qemu:commandline> > <qemu:arg value='-device'/> > <qemu:arg value='vfio-pci,host=05:00.0,bus=pcie.0'/> > </qemu:commandline> > > but I insist on running the VM as non-root, and if I got that right I > need to configure at least one vfio device (or memory locking) in > order for libvirt to set a proper RLIMIT_MEMLOCK value. > > Any help would be be appreciated. For now at least, you'll need to let it plug into the pci-bridge device pci.2 (which, as you've found, libvirt will automatically find when you don't specify any address). Unfortunately that doesn't do you much good, since that particular device you're assigning actually requires that it be plugged into the PCIe bus. I'm wondering as I type if possibly we could relax the enforcement of the "PCI only" rule such that we allow explicitly placing any device on any type of bus, but only auto-assign to a plain PCI slot. That may be a reasonable compromise until qemu has the required new device/controller introspection info available. _______________________________________________ libvirt-users mailing list libvirt-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvirt-users
