On Sat, Nov 09, 2019 at 09:41:03AM -0800, Jakub Kicinski wrote: > On Fri, 8 Nov 2019 20:57:08 -0400, Jason Gunthorpe wrote: > > On Fri, Nov 08, 2019 at 10:48:31PM +0000, Parav Pandit wrote: > > > We should be creating 3 different buses, instead of mdev bus being de-multiplexer of that? > > > > > > Hence, depending the device flavour specified, create such device on right bus? > > > > > > For example, > > > $ devlink create subdev pci/0000:05:00.0 flavour virtio name foo subdev_id 1 > > > $ devlink create subdev pci/0000:05:00.0 flavour mdev <uuid> subdev_id 2 > > > $ devlink create subdev pci/0000:05:00.0 flavour mlx5 id 1 subdev_id 3 > > > > I like the idea of specifying what kind of interface you want at sub > > device creation time. It fits the driver model pretty well and doesn't > > require abusing the vfio mdev for binding to a netdev driver. > > Aren't the HW resources spun out in all three cases exactly identical? Exactly? No, not really. The only constant is that some chunk of the BAR is dedicated to this subedv. The BAR is flexible, so a BAR chunk configured for virtio is not going to support mlx5 mode. Aside from that, there are other differences ie - mlx5 does not need a dedicated set of MSI-X's while other modes do. There are fewer MSI-X's than SF's, so managing this is important for the admin. Even in modes which are very similar, like mlx5 vs mdev-vfio, the HW still has to be configured to provide global DMA isolation on the NIC for vfio as the IOMMU cannot be involved. This is extra overhead and should not be activated unless vfio is being used. .. and finally the driver core does not support a 'multiple-inheritance' like idea, so we can't have a 'foo_device' that is three different things. So somehow the 'flavour' of the 'struct device' has to be exposed to userspace, and it is best if this is done at device creation time so the BAR region and HW can be setup once and we don't have to have complex reconfiguration flows. Jason
