On Thu, Jun 08, 2023 at 04:05:54PM -0400, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@xxxxxxxxxx>
>
> We would like to enable the use of siw on top of a VPN that is
> constructed and managed via a tun device. That hasn't worked up
> until now because ARPHRD_NONE devices (such as tun devices) have
> no GID for the RDMA/core to look up.
>
> But it turns out that the egress device has already been picked for
> us -- no GID is necessary. addr_handler() just has to do the right
> thing with it.
>
> Suggested-by: Jason Gunthorpe <jgg@xxxxxxxxxx>
> Signed-off-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
> ---
> drivers/infiniband/core/cma.c | 27 ++++++++++++++++++++++-----
> 1 file changed, 22 insertions(+), 5 deletions(-)
>
> Further testing convinced me of the necessity of confirming that
> the ndev and ib_device are properly related. This version works
> on systems with multiple RDMA devices present.
>
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 56e568fcd32b..44ef0539957a 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -686,30 +686,47 @@ cma_validate_port(struct ib_device *device, u32 port,
> struct rdma_id_private *id_priv)
> {
> struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
> + const struct ib_gid_attr *sgid_attr = ERR_PTR(-ENODEV);
> int bound_if_index = dev_addr->bound_dev_if;
> - const struct ib_gid_attr *sgid_attr;
> int dev_type = dev_addr->dev_type;
> struct net_device *ndev = NULL;
>
> if (!rdma_dev_access_netns(device, id_priv->id.route.addr.dev_addr.net))
> - return ERR_PTR(-ENODEV);
> + goto out;
> +
> + if (rdma_protocol_iwarp(device, port)) {
> + struct ib_device *base_dev;
> +
> + ndev = dev_get_by_index(dev_addr->net, bound_if_index);
> + if (!ndev)
> + goto out;
> + base_dev = ib_device_get_by_netdev(ndev, RDMA_DRIVER_UNKNOWN);
> + if (base_dev)
> + ib_device_put(base_dev);
> + dev_put(ndev);
> +
> + if (device == base_dev)
> + sgid_attr = rdma_get_gid_attr(device, port, 0);
> + goto out;
> + }
Oy, this is kind of ugly - did you look at having the iwarp side
properly set the ndev in the sgid_attrs instead?
Then you can just check the sgid_attrs->ndev->'net && bound_if_indx' == dev_addr
Jason
