On Fri, 8 Nov 2019 16:12:53 -0400 Jason Gunthorpe <jgg@xxxxxxxx> wrote: > On Fri, Nov 08, 2019 at 11:12:38AM -0800, Jakub Kicinski wrote: > > On Fri, 8 Nov 2019 15:40:22 +0000, Parav Pandit wrote: > > > > The new intel driver has been having a very similar discussion about how to > > > > model their 'multi function device' ie to bind RDMA and other drivers to a > > > > shared PCI function, and I think that discussion settled on adding a new bus? > > > > > > > > Really these things are all very similar, it would be nice to have a clear > > > > methodology on how to use the device core if a single PCI device is split by > > > > software into multiple different functional units and attached to different > > > > driver instances. > > > > > > > > Currently there is alot of hacking in this area.. And a consistent scheme > > > > might resolve the ugliness with the dma_ops wrappers. > > > > > > > > We already have the 'mfd' stuff to support splitting platform devices, maybe > > > > we need to create a 'pci-mfd' to support splitting PCI devices? > > > > > > > > I'm not really clear how mfd and mdev relate, I always thought mdev was > > > > strongly linked to vfio. > > > > > > > > > > Mdev at beginning was strongly linked to vfio, but as I mentioned > > > above it is addressing more use case. > > > > > > I observed that discussion, but was not sure of extending mdev further. > > > > > > One way to do for Intel drivers to do is after series [9]. > > > Where PCI driver says, MDEV_CLASS_ID_I40_FOO > > > RDMA driver mdev_register_driver(), matches on it and does the probe(). > > > > Yup, FWIW to me the benefit of reusing mdevs for the Intel case vs > > muddying the purpose of mdevs is not a clear trade off. > > IMHO, mdev has amdev_parent_ops structure clearly intended to link it > to vfio, so using a mdev for something not related to vfio seems like > a poor choice. Unless there's some opposition, I'm intended to queue this for v5.5: https://www.spinics.net/lists/kvm/msg199613.html mdev has started out as tied to vfio, but at it's core, it's just a device life cycle infrastructure with callbacks between bus drivers and vendor devices. If virtio is on the wrong path with the above series, please speak up. Thanks, Alex > I suppose this series is the start and we will eventually see the > mlx5's mdev_parent_ops filled in to support vfio - but *right now* > this looks identical to the problem most of the RDMA capable net > drivers have splitting into a 'core' and a 'function' > > > IMHO MFD should be of more natural use for Intel, since it's about > > providing different functionality rather than virtual slices of the > > same device. > > I don't think the 'different functionality' should matter much. > > Generally these multi-function drivers are build some some common > 'core' language like queues interrupts, BAR space, etc and then these > common things can be specialized into netdev, rdma, scsi, etc. So we > see a general rough design with a core layer managing the raw HW then > drivers on top of that (including netdev) using that API. > > The actual layering doesn't come through in the driver model, > generally people put all the core stuff in with the netdev and then > try and shuffle the netdev around as the 'handle' for that core API. > > These SFs are pretty similar in that the core physical driver > continues to provide some software API support to the SF children (at > least for mlx it is a small API) > > For instance mdev has no generic way to learn the BAR struct > resources, so there is some extra API around the side that does this - > in this series it is done by hackily co-opting the drvdata to > something owned by the struct device instead of the device_driver and > using that to access the API surface on 'struct mlx5_sf *', which > includes the BAR info and so forth. > > This is probably the main difference from MFD. At least the few > drivers I looked at, did not try and expose an SW API from the 'core' > to the 'part', everything was usual generic driver resource stuff. > > Jason
