> Subject: Re: [PATCH 07/22] RDMA/irdma: Register an auxiliary driver and > implement private channel OPs > > On Wed, Jan 27, 2021 at 12:41:41AM +0000, Saleem, Shiraz wrote: > > > Subject: Re: [PATCH 07/22] RDMA/irdma: Register an auxiliary driver > > > and implement private channel OPs > > > > > > On Tue, Jan 26, 2021 at 12:42:16AM +0000, Saleem, Shiraz wrote: > > > > > > > I think this essentially means doing away with .open/.close piece. > > > > > > Yes, that too, and probably the FSM as well. > > > > > > > Or are you saying that is ok? Yes we had a discussion in the past > > > > and I thought we concluded. But maybe I misunderstood. > > > > > > > > https://lore.kernel.org/linux-rdma/9DD61F30A802C4429A01CA4200E302A > > > > 7DCD 4FD03@xxxxxxxxxxxxxxxxxxxxxxxxxxxx/ > > > > > > Well, having now seen how aux bus ended up and the way it effected > > > the > > > mlx5 driver, I am more firmly of the opinion this needs to be fixed. > > > It is extremly hard to get everything right with two different registration > schemes running around. > > > > > > You never answered my question: > > > > Sorry I missed it. > > > > > > > Still, you need to be able to cope with the user unbinding your > > > > drivers in any order via sysfs. What happens to the VFs when the > > > > PF is unbound and releases whatever resources? This is where the > > > > broadcom driver ran into troubles.. > > > > > > ? > > > > echo -n "ice.intel_rdma.0" > /sys/bus/auxiliary/drivers/irdma/unbind ??? > > > > That I believe will trigger a drv.remove() on the rdma PF side which > > require the rdma VFs to go down. > > > > Yes, we currently have a requirement the aux rdma PF driver remain > > inited at least to .probe() for VFs to survive. > > > > We are doing internal review, but it appears we could potentially get rid of the > .open/.close callbacks. > > And its associated FSM in ice. > > > > But if we remove peer_register/unregister, how do we synchronize > > between say unload of the rdma driver and netdev driver stop accessing the priv > channel iidc_peer_ops that it uses to send events to rdma? > > And here we are returning to square one of intended usage of aux bus. > Your driver should be structured to have PCI core logic that will represent physical > device and many small sub-devices with their respective drivers. Even with another core PCI driver, there still needs to be private communication channel between the aux rdma driver and this PCI driver to pass things like QoS updates. The split of the PCI core device into aux devices is a design choice. Not sure what that has to do with the usage of bus. > > ETH is another sub-device that shouldn't talk directly to the RDMA. > ETH should talk indirectly to RDMA driver via the core PCI driver? And how does that solve this?
