On Thu, 2024-04-04 at 21:44 +0800, Wen Gu wrote: > > On 2024/4/4 19:27, Niklas Schnelle wrote: > > On Thu, 2024-04-04 at 18:20 +0800, Wen Gu wrote: > > > > > > On 2024/4/4 01:20, Gerd Bayer wrote: > > > > On Sun, 2024-03-24 at 21:55 +0800, Wen Gu wrote: > > > > > > > > When I instrumented this to see, why I still see tons of my other > > > > temporary instrumentation messages from the "ism" driver, I found that > > > > in my setup loopback-ism is used rather infrequently. > > > > > > > > I suspect this is due to how the SMC proposals are constructed in > > > > net/smc/af_smc.c and net/smc/smc_pnet.c - and later evaluated in > > > > smc_check_ism_v2_match() - where there is a first-come-first-serve > > > > selection. > > > > > > > > I wonder if one should change that to favour loopback-ism over "real" > > > > ISM devices - and how this could be achieved elegantly. > > > > > > > > Just some food for thought... Probably little you can do on x86. > > > > > > > > > > Yes, it is about the priority of available ISM devices, and now it > > > is decided by their order in the smcd_dev_list. The later registered > > > ISMv2 devices(without pnetid) will be added to the beginning of the > > > list (see smcd_register_dev()). So there is a probability that > > > loopback-ism will not be ranked first, since it is added into list > > > earlier during smc_init(). > > > > > > If we have the runtime switch of loopback-ism, we can re-active the > > > loopback-ism, that make it be re-added into the beginning of the dev > > > list and be chosen first. Or a new netlink command to adjust the slot > > > order of available ISM devices in the list. As we discussed before, > > > that could be tasks in stage 1 or stage 2. > > > > > > Thanks! > > > > Maybe when adding the ISM devices we could instead make sure that all > > ISM devices are added after loopback and loopback is added in the > > beginning. I think loopback should always be preferred and would > > consider it a bug if it isn't faster too. Between virtio-ism and ISM it > > may be less clear so maybe for stage 2 we would want a priority setting > > and then insert ordered by priority. Thoughts? > I have no objection. If we all agree, I will keep it at the beginning of the list. > > Thanks! > > I think this is a decision that needs to be made by the SMC maintainers and Wenjia will be back next week. That said the current code basically prioritizes ISM devices without that having been a conscious and documented decision. Also note that due to hotplug an LPAR could be using loopback-ism happily until suddenly an ISM device is hot plugged and new connections suddenly switch to ISM.
