On 25.09.23 17:18, Dust Li wrote: >> Hello Wen Gu, >> >> thank you for adding the Kconfig, so the distributions can decide when to offer this feature. >> >> I propose you add some kind of runtime switch as well. Not every user who loads the SMC module >> may want to exploit smcd-loopback. Especially in native environements without containers. >> >> If no RoCE interfaces or no ISM interfaces exist, the respective handling is skipped in SMC. >> If loopback is always created unconditionally, there is no way to opt-out. > Hi Sandy, > > After talking to Wen Gu offline, I think the real issue here might be > we don't have an abstract layer in SMC, something like net/core/dev.c > > Without this, we cannot do: > > 1. Enable/disable those devices dynamically > Currently, If we want to disable a SMC-R device to communicate with > others, we need to refer to 'ip link set dev xxx down' to disable the > netdevice, then Infiniband subsystem will notify SMC that the state of > the IB device has changed. We cannot explicitly choose not to use some > specific IB/RoCE devices without disable totally. > If the loopback device need to support enable/disable itself, I > think it might be better to enable this feature for all SMC devices. > > 2. Do statistics per device > Now, we have to relay on IB/RoCE devices' hardware statistics to see > how many packets/bytes we have sent through this device. > > Both the above issues get worse when the IB/RoCE device is shared by SMC > and userspace RDMA applications. If SMC-R and userspace RDMA applications > run at the same time, we can't enable the device to run userspace RDMA > applications while block it from running SMC. For statistics, we cannot > tell how many packets/bytes were sent by SMC and how many were sent by > userspace RDMA applications. > > So I think those are better to support in the SMC layer. > > Best regards! > Dust Thank you very much for your considerations. I also think a generic handling of these requirements in the smc layer would be best. Especially, if we want to add virtio-ism support soon. There we will face the same issues again. Let's hear what others think about this.
