> -----Original Message----- > From: Jiri Pirko <jiri@xxxxxxxxxxx> > > Fri, Nov 08, 2019 at 08:06:40PM CET, jakub.kicinski@xxxxxxxxxxxxx wrote: > >On Fri, 8 Nov 2019 13:12:33 +0100, Jiri Pirko wrote: > >> Thu, Nov 07, 2019 at 09:32:34PM CET, jakub.kicinski@xxxxxxxxxxxxx > wrote: > >> >On Thu, 7 Nov 2019 10:04:48 -0600, Parav Pandit wrote: > >> >> Mellanox sub function capability allows users to create several > >> >> hundreds of networking and/or rdma devices without depending on PCI > SR-IOV support. > >> > > >> >You call the new port type "sub function" but the devlink port > >> >flavour is mdev. > >> > > >> >As I'm sure you remember you nacked my patches exposing NFP's PCI > >> >sub functions which are just regions of the BAR without any mdev > >> >capability. Am I in the clear to repost those now? Jiri? > >> > >> Well question is, if it makes sense to have SFs without having them > >> as mdev? I mean, we discussed the modelling thoroughtly and > >> eventually we realized that in order to model this correctly, we need SFs on > "a bus". > >> Originally we were thinking about custom bus, but mdev is already > >> there to handle this. > > > >But the "main/real" port is not a mdev in your case. NFP is like mlx4. > >It has one PCI PF for multiple ports. > > I don't see how relevant the number of PFs-vs-uplink_ports is. > > > > > >> Our SFs are also just regions of the BAR, same thing as you have. > >> > >> Can't you do the same for nfp SFs? > >> Then the "mdev" flavour is enough for all. > > > >Absolutely not. > > > >Why not make the main device of mlx5 a mdev, too, if that's acceptable. > >There's (a) long precedence for multiple ports on one PCI PF in > >networking devices, (b) plenty deployed software which depend on the > >main devices hanging off the PCI PF directly. > > > >The point of mdevs is being able to sign them to VFs or run DPDK on > >them (map to user space). > > > >For normal devices existing sysfs hierarchy were one device has > >multiple children of a certain class, without a bus and a separate > >driver is perfectly fine. Do you think we should also slice all serial > >chips into mdevs if they have multiple lines. > > > >Exactly as I predicted much confusion about what's being achieved here, > >heh :) > > Please let me understand how your device is different. > Originally Parav didn't want to have mlx5 subfunctions as mdev. He wanted to > have them tight to the same pci device as the pf. No difference from what you > describe you want. > However while we thought about how to fit things in, how to > handle na phys_port_name, how to see things in sysfs we came up with an idea > of a dedicated bus. We took it upstream and people suggested to use mdev bus > for this. > You are right. We considered multiple ports approach, followed by subdevices and mfd. Around that time mdev was being proposed that can address current and future VM/userspace usecases using one way to lifecycle the devices. > Parav, please correct me if I'm wrong but I don't think where is a plan to push > SFs into VM or to userspace as Jakub expects, right? With this series - certainly not. In future, if mdev to be used by via vfio/VM framework, why should we prevent it (ofcourse after implementing necessary isolation method)?
