> On Jun 12, 2023, at 10:40 AM, Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
>
> On Thu, Jun 08, 2023 at 04:05:54PM -0400, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@xxxxxxxxxx>
>>
>> We would like to enable the use of siw on top of a VPN that is
>> constructed and managed via a tun device. That hasn't worked up
>> until now because ARPHRD_NONE devices (such as tun devices) have
>> no GID for the RDMA/core to look up.
>>
>> But it turns out that the egress device has already been picked for
>> us -- no GID is necessary. addr_handler() just has to do the right
>> thing with it.
>>
>> Suggested-by: Jason Gunthorpe <jgg@xxxxxxxxxx>
>> Signed-off-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
>> ---
>> drivers/infiniband/core/cma.c | 27 ++++++++++++++++++++++-----
>> 1 file changed, 22 insertions(+), 5 deletions(-)
>>
>> Further testing convinced me of the necessity of confirming that
>> the ndev and ib_device are properly related. This version works
>> on systems with multiple RDMA devices present.
>>
>>
>> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
>> index 56e568fcd32b..44ef0539957a 100644
>> --- a/drivers/infiniband/core/cma.c
>> +++ b/drivers/infiniband/core/cma.c
>> @@ -686,30 +686,47 @@ cma_validate_port(struct ib_device *device, u32 port,
>> struct rdma_id_private *id_priv)
>> {
>> struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
>> + const struct ib_gid_attr *sgid_attr = ERR_PTR(-ENODEV);
>> int bound_if_index = dev_addr->bound_dev_if;
>> - const struct ib_gid_attr *sgid_attr;
>> int dev_type = dev_addr->dev_type;
>> struct net_device *ndev = NULL;
>>
>> if (!rdma_dev_access_netns(device, id_priv->id.route.addr.dev_addr.net))
>> - return ERR_PTR(-ENODEV);
>> + goto out;
>> +
>> + if (rdma_protocol_iwarp(device, port)) {
>> + struct ib_device *base_dev;
>> +
>> + ndev = dev_get_by_index(dev_addr->net, bound_if_index);
>> + if (!ndev)
>> + goto out;
>> + base_dev = ib_device_get_by_netdev(ndev, RDMA_DRIVER_UNKNOWN);
>> + if (base_dev)
>> + ib_device_put(base_dev);
>> + dev_put(ndev);
>> +
>> + if (device == base_dev)
>> + sgid_attr = rdma_get_gid_attr(device, port, 0);
>> + goto out;
>> + }
>
> Oy, this is kind of ugly -
Yeah, not 100% elegant.
> did you look at having the iwarp side
> properly set the ndev in the sgid_attrs instead?
Interesting. I'll have a look.
> Then you can just check the sgid_attrs->ndev->'net && bound_if_indx' == dev_addr
--
Chuck Lever
