-----"Chuck Lever" <chuck.lever@xxxxxxxxxx> wrote: ----- >To: "linux-rdma" <linux-rdma@xxxxxxxxxxxxxxx> >From: "Chuck Lever" <chuck.lever@xxxxxxxxxx> >Date: 02/11/2021 08:38PM >Cc: "Benjamin Coddington" <bcodding@xxxxxxxxxx> >Subject: [EXTERNAL] directing soft iWARP traffic through a secure >tunnel > >Hi- > >This might sound crazy, but bear with me. > >The NFS community is starting to hold virtual interoperability >testing >events to replace our in-person events that are not feasible due to >pandemic-related travel restrictions. I'm told other communities have >started doing the same. > >The virtual event is being held on a private network that is set up >using OpenVPN across a large geographical area. I attach my test >systems to the VPN to access test systems run by others at other >companies. > >We'd like to continue to include NFS/RDMA testing at these events. >This means either RoCEv2 or iWARP, since obviously we can't create >an ad hoc wide-area InfiniBand infrastructure. > >Because the VPN is operating over long distances, we've decided to >start with iWARP. However, we are stumbling when it comes to >directing >the siw driver's traffic onto the tun0 device: > >[root@oracle-100 ~]# rdma link add siw0 type siw netdev tun0 >error: Invalid argument >[root@oracle-100 ~]# > >Has anyone else tried to do this, and what was the approach? Or does >siw not yet have this capability? > >Thanks in advance. > > After some very little ad-hoc patching of siw and RDMA core, we get RDMA connectivity over the VPN. For siw, all what is needed is to enable attaching to other netdev's than type ARPHRD_ETHER, in particular allowing ARPHRD_NONE. It is questionable, if software RDMA drivers need any such limitation, if running on top of TCP or UDP. The RDMA core currently does not accept a zero gid, which would be the result of attaching to a tunnel w/o hardware address. I tentatively disabled that code for tunnel devices (simply by allowing a zero gid for device names starting with 'tun'), but a serious solution would be different. I think enabling software RDMA over TUN/TAP devices makes a lot of sense, as exemplified by enabling this virtual NFS/RDMA test event. If we agree on that, next step would be to come up with the right way to do it. Would it make sense to start from the rdma_link_layer enum, which already knows type IB_LINK_LAYER_UNSPECIFIED ? RDMA drivers can report the link type via device->ops.get_link_layer - see rdma_port_get_link_layer(). Asking the RDMA core experts - would a gid of zero have any side effects or bad implications, since that ID is by no means unique? Many thanks, Bernard.
