在 2023/6/12 8:19, Chuck Lever III 写道:
On Jun 10, 2023, at 8:05 AM, Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
On Fri, Jun 09, 2023 at 04:49:49PM -0400, Tom Talpey wrote:
On 6/7/2023 3:43 PM, Chuck Lever wrote:
From: Chuck Lever <chuck.lever@xxxxxxxxxx>
We would like to enable the use of siw on top of a VPN that is
constructed and managed via a tun device. That hasn't worked up
until now because ARPHRD_NONE devices (such as tun devices) have
no GID for the RDMA/core to look up.
But it turns out that the egress device has already been picked for
us. addr_handler() just has to do the right thing with it.
Tested with siw and qedr, both initiator and target.
Suggested-by: Jason Gunthorpe <jgg@xxxxxxxxxx>
Signed-off-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
---
drivers/infiniband/core/cma.c | 3 +++
1 file changed, 3 insertions(+)
This of course needs broader testing, but it seems to work, and it's
a little nicer than "if (dev_type == ARPHRD_NONE)".
One thing I discovered is that the NFS/RDMA server implementation
does not deal at all with more than one RDMA device on the system.
I will address that with an ib_client; SunRPC patches forthcoming.
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 56e568fcd32b..c9a2bdb49e3c 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -694,6 +694,9 @@ cma_validate_port(struct ib_device *device, u32 port,
if (!rdma_dev_access_netns(device, id_priv->id.route.addr.dev_addr.net))
return ERR_PTR(-ENODEV);
+ if (rdma_protocol_iwarp(device, port))
+ return rdma_get_gid_attr(device, port, 0);
This might work, but I'm skeptical of the conditional. There's nothing
special about iWARP that says a GID should come from the outgoing device
mac. And, other protocols without IB GID addressing won't benefit.
The GID represents the source address, so it better come from the
outgoing device or something is really wrong.
iWARP gets a conditional because iwarp always has a single GID, other
devices do not work that way.
If rdma_get_gid_attr() works for rxe, perhaps we could change
the conditional to match on only software-emulated devices.
drivers/infiniband/sw/rxe/rxe_net.c:
453 struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
454 int paylen, struct rxe_pkt_info *pkt)
455 {
456 unsigned int hdr_len;
457 struct sk_buff *skb = NULL;
458 struct net_device *ndev;
459 const struct ib_gid_attr *attr;
460 const int port_num = 1;
461
462 attr = rdma_get_gid_attr(&rxe->ib_dev, port_num,
av->grh.sgid_index);
463 if (IS_ERR(attr))
464 return NULL;
465
...
Zhu Yanjun
--
Chuck Lever
