> On Wed, Jul 26, 2017 at 09:05:03AM -0500, Steve Wise wrote: > > > > I think you should change this to create the RDMA device when the > > > module is installed and the hardware is present.. > > > > > > > Not gonna happen. cxgb4 doesn't setup the queues, rss, irq mappings, etc, until > > an interface is brought up. So iw_cxgb4 cannot initialize and register with the > > rdma core until that happens. > > But it doesn't tear all that stuff down on ifdown? hmm. I need to relook at this. I'm very sure the intent was offload connections do not get torn down if the link goes down. > > > > Basically, it is very hard to start a RDMA daemon and not have it race > > > with something and randomly fail to start properly the more hotpluggy > > > things are. > > > > I think these races exist today, no? Or is this patch series introducing the > > races? The iwpmd does not need the providers registered at the time it starts. > > It will discover new iwarp providers as they initialize. > > There are races today, particularly with systemd, but I suspect they > are slightly different.. That doesn't mean we should ignore them :) > > In particular the rdma.target approach loaded all the modules, even if > there wasn't hardware support, so when the cxgb4 RDMA device is > created everything is already to go. We still have races, but they are > probably smaller > > Now, we don't even plug the modules until the RDMA device appears, so > we have broader possibilities for racing.. > > Overall it makes it hard to fit into the ideal system configuration > where RDMA sets up before network-pre.target when a RDMA device > doesn't even plug in until after network.target. > > > Can you think of other ways to address your concerns? > > Maybe we should have udev trigger loading iwarp services on the cxgb4 > ethernet device? At least the races then become more like what we had > before. Are other roce/iwarp devices like this? > I'm not sure. I would be interested to hear what Intel does. +Chien + Mustafa Steve. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
