On Thu, Mar 15, 2018 at 08:59:41AM -0600, Alex Williamson wrote: > On Thu, 15 Mar 2018 14:33:55 +0000 > Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: > > > On Thu, Mar 15, 2018 at 07:54:47PM +0530, Shivaprasad G Bhat wrote: > > > > > > > > > On 03/15/2018 03:31 PM, Daniel P. Berrangé wrote: > > > > On Wed, Mar 14, 2018 at 10:44:30PM +0530, Shivaprasad G Bhat wrote: > > > > > Hi All, > > > > > > > > > > I have revisited/rewritten my previously posted patches. Here is > > > > > the RFC. Since this patchset is a complete rewrite, I am starting > > > > > with v1 here. > > > > > > > > > > The semantics is as discussed before > > > > > https://www.redhat.com/archives/libvir-list/2016-April/msg01057.html > > > > > > > > > > As I went on to refactor the code to support multifunction virtio devices, > > > > > I realised the abort/cleanup path would be a nightmare there, in case of > > > > > failures. So, dropped that attempt. The current RFC limits to the real > > > > > practical use cases of Multifunction PCI hostdevices. All new test code > > > > > to support multifunction PCI hostdevices and test cases are added to > > > > > prove the functionality. > > > > I guess I'm not really understanding the use case here. With SRIOV > > > > devices, you can already choose between assigning either the physical > > > > function (which gives the guest access to all virtual functions), or > > Say what? If a guest is assigned a PF, they get the PF, they don't get > to enable SR-IOV to also get the VFs. But SR-IOV and multifunction are > far from synonymous nor is SR-IOV ubiquitous to all use cases, so I > don't know why we're bringing SR-IOV into this discussion. > > > > > to assign an arbitrary set of individiual functions to various guests. > > > > Why do we need to be able to list many <hostdev> at the same time > > > > when hotplugging to assign multiple functions. > > > > > > > > Basically can you provide a full description of the problem you are > > > > trying to solve and why existing functionality isn't sufficient. > > > > > > Hi Daniel, > > > > > > This is for cards which may not necessarily be networking cards. Or may be a > > > mix of > > > networking and storage. > > > > > > Suppose, user has below card > > > 0005:01:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) > > > (rev 10) > > > 0005:01:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) > > > (rev 10) > > > 0005:01:00.2 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) > > > (rev 10) > > > 0005:01:00.3 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) > > > (rev 10) > > > 0005:01:00.4 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator > > > (Lancer) (rev 10) > > > 0005:01:00.5 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator > > > (Lancer) (rev 10) > > > > Ok, so this is a device with many functions, but which isn't SRIOV > > based, and the goal is to assign the physical device to the guest, > > such that guest has all functions available. > > > > > If user wants to hotplug this card to guest, He has to detach all the > > > functions from host driver, > > > then hotplug 0005:01:00.0, 0005:01:00.1, so on individually. But, today with > > > each hotplug > > > of the function, each <hostdev> goes to different guest slot. Whereas, PCI > > > requires all of > > > them to be on the same slot. This is not supported on libvirt today. > > > > > > The multifunction cards cant be hotplugged to guest today with the > > > individual > > > <hostdev>, as the operation is queued by qemu till the function zero of > > > guest slot is > > > hotplugged. On function zero hotplug, the qemu sends out the event to guest > > > for device probing where all the previously hotplugged functions from the > > > same slot are discovered. So, grouping the <hostdev>s within the <devices> > > > would become necessary to make the whole thing a single operation. > > > > So IIUC, from the patches, if the user wants to assign the physical > > device to the guest, they would need to provide XML that looked like > > this to the virDomainAttachDevice() method: > > > > <devices> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x0'/> > > </source> > > </hostdev> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x1'/> > > </source> > > </hostdev> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x2'/> > > </source> > > </hostdev> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x3'/> > > </source> > > </hostdev> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x4'/> > > </source> > > </hostdev> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x5'/> > > </source> > > </hostdev> > > </devices> > > > > > > Where as if the device were SRIOV based, they would only have to > > provide > > > > <device> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x0'/> > > </source> > > </hostdev> > > </device> > > > > for the guest to get access to all functions. > > Since when has this been the case? (nit, the example is domain=0x5, > bus=0x1,...) > > > I find this difference in behaviour and approach really unpleasant. > > > > I think that they user should only need to provide the the address > > of the physical device, in both cases. At most perhaps we need a > > new attribute multifunction="on" on the source address to tell > > libvirt that it should attach all the functions, not just the > > first > > > > <device> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0x05' slot='0x1' function='0x0' mutlifunction="on"/> > > </source> > > </hostdev> > > </device> > > Neither really bothers me, but I'm confused by the claimed existing > handling of SR-IOV. Either you're assigning a PF and SR-IOV is > irrelevant and unavailable to the guest or you're assigning a VF and, > well, SR-IOV is still mostly irrelevant to libvirt unless someone > decides to assign the PF hosting the VF or libvirt needs to do VF > configuration via the PF. Thanks, Hmm, could be a mis-understanding then. I was under the belief that when you assign the PF or a SRIOV device to the guest, all the VFs obviously disappear from the host due to driver being unloaded. The guest now has the PF, and would have the option to enable VFs too if it so desired, just as the host had option for when it owned the PF. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
