On Wed, Jul 26, 2017 at 09:05:03AM -0500, Steve Wise wrote: > > I think you should change this to create the RDMA device when the > > module is installed and the hardware is present.. > > > > Not gonna happen. cxgb4 doesn't setup the queues, rss, irq mappings, etc, until > an interface is brought up. So iw_cxgb4 cannot initialize and register with the > rdma core until that happens. But it doesn't tear all that stuff down on ifdown? > > Basically, it is very hard to start a RDMA daemon and not have it race > > with something and randomly fail to start properly the more hotpluggy > > things are. > > I think these races exist today, no? Or is this patch series introducing the > races? The iwpmd does not need the providers registered at the time it starts. > It will discover new iwarp providers as they initialize. There are races today, particularly with systemd, but I suspect they are slightly different.. That doesn't mean we should ignore them :) In particular the rdma.target approach loaded all the modules, even if there wasn't hardware support, so when the cxgb4 RDMA device is created everything is already to go. We still have races, but they are probably smaller Now, we don't even plug the modules until the RDMA device appears, so we have broader possibilities for racing.. Overall it makes it hard to fit into the ideal system configuration where RDMA sets up before network-pre.target when a RDMA device doesn't even plug in until after network.target. > Can you think of other ways to address your concerns? Maybe we should have udev trigger loading iwarp services on the cxgb4 ethernet device? At least the races then become more like what we had before. Are other roce/iwarp devices like this? Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
