On 1/28/2021 11:02 PM, Alex Williamson wrote:
On Thu, 28 Jan 2021 17:29:30 +0100
Cornelia Huck <cohuck@xxxxxxxxxx> wrote:
On Tue, 26 Jan 2021 15:27:43 +0200
Max Gurtovoy <mgurtovoy@xxxxxxxxxx> wrote:
On 1/26/2021 5:34 AM, Alex Williamson wrote:
On Mon, 25 Jan 2021 20:45:22 -0400
Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
On Mon, Jan 25, 2021 at 04:31:51PM -0700, Alex Williamson wrote:
extensions potentially break vendor drivers, etc. We're only even hand
waving that existing device specific support could be farmed out to new
device specific drivers without even going to the effort to prove that.
This is a RFC, not a complete patch series. The RFC is to get feedback
on the general design before everyone comits alot of resources and
positions get dug in.
Do you really think the existing device specific support would be a
problem to lift? It already looks pretty clean with the
vfio_pci_regops, looks easy enough to lift to the parent.
So far the TODOs rather mask the dirty little secrets of the
extension rather than showing how a vendor derived driver needs to
root around in struct vfio_pci_device to do something useful, so
probably porting actual device specific support rather than further
hand waving would be more helpful.
It would be helpful to get actual feedback on the high level design -
someting like this was already tried in May and didn't go anywhere -
are you surprised that we are reluctant to commit alot of resources
doing a complete job just to have it go nowhere again?
That's not really what I'm getting from your feedback, indicating
vfio-pci is essentially done, the mlx stub driver should be enough to
see the direction, and additional concerns can be handled with TODO
comments. Sorry if this is not construed as actual feedback, I think
both Connie and I are making an effort to understand this and being
hampered by lack of a clear api or a vendor driver that's anything more
than vfio-pci plus an aux bus interface. Thanks,
I think I got the main idea and I'll try to summarize it:
The separation to vfio-pci.ko and vfio-pci-core.ko is acceptable, and we
do need it to be able to create vendor-vfio-pci.ko driver in the future
to include vendor special souse inside.
One other thing I'd like to bring up: What needs to be done in
userspace? Does a userspace driver like QEMU need changes to actually
exploit this? Does management software like libvirt need to be involved
in decision making, or does it just need to provide the knobs to make
the driver configurable?
I'm still pretty nervous about the userspace aspect of this as well.
QEMU and other actual vfio drivers are probably the least affected,
at least for QEMU, it'll happily open any device that has a pointer to
an IOMMU group that's reflected as a vfio group device. Tools like
libvirt, on the other hand, actually do driver binding and we need to
consider how they make driver decisions. Jason suggested that the
vfio-pci driver ought to be only spec compliant behavior, which sounds
like some deprecation process of splitting out the IGD, NVLink, zpci,
etc. features into sub-drivers and eventually removing that device
specific support from vfio-pci. Would we expect libvirt to know, "this
is an 8086 graphics device, try to bind it to vfio-pci-igd" or "uname
-m says we're running on s390, try to bind it to vfio-zpci"? Maybe we
expect derived drivers to only bind to devices they recognize, so
libvirt could blindly try a whole chain of drivers, ending in vfio-pci.
Obviously if we have competing drivers that support the same device in
different ways, that quickly falls apart.
I think we can leave common arch specific stuff, such as s390 (IIUC) in
the core driver. And only create vfio_pci drivers for
vendor/device/subvendor specific stuff.
Also, the competing drivers issue can also happen today, right ? after
adding new_id to vfio_pci I don't know how linux will behave if we'll
plug new device with same id to the system. which driver will probe it ?
I don't really afraid of competing drivers since we can ask from vendor
vfio pci_drivers to add vendor_id, device_id, subsystem_vendor and
subsystem_device so we won't have this problem. I don't think that there
will be 2 drivers that drive the same device with these 4 ids.
Userspace tool can have a map of ids to drivers and bind the device to
the right vfio-pci vendor driver if it has one. if not, bind to vfio_pci.ko.
Libvirt could also expand its available driver models for the user to
specify a variant, I'd support that for overriding a choice that libvirt
might make otherwise, but forcing the user to know this information is
just passing the buck.
We can add a code to libvirt as mentioned above.
Some derived drivers could probably actually include device IDs rather
than only relying on dynamic ids, but then we get into the problem that
we're competing with native host driver for a device. The aux bus
example here is essentially the least troublesome variation since it
works in conjunction with the native host driver rather than replacing
it. Thanks,
same competition after we add new_id to vfio_pci, right ?
Alex
A pointer to needed additions to libvirt will be awsome (or any other hint).
I'll send the V2 soon and then move to libvirt.
-Max.
