Creating rxe device on top of vlan interface will create a non-functional device
that has an empty gids table and can't be used for rdma cm communication.
This is caused by the logic in enum_all_gids_of_dev_cb()/is_eth_port_of_netdev(),
which only considers networks connected to "upper devices" of the configured
network device, resulting in an empty set of gids for a vlan interface,
and attempts to connect via this rdma device fail in cm_init_av_for_response
because no gids can be resolved.
apparently, this behavior was implemented to fit the HW-RoCE devices that create
RoCE device per port, therefore RXE must behave the same like HW-RoCE devices
and create rxe device per real device only.
In order to communicate via a vlan interface, the user must use the gid index of
the vlan address instead of creating rxe over vlan.
Signed-off-by: Mohammad Heib <goody698@xxxxxxxxx>
---
drivers/infiniband/sw/rxe/rxe.c | 6 ++++++
drivers/infiniband/sw/rxe/rxe_sysfs.c | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c
index 5642eefb4ba1..d2076aa7a732 100644
--- a/drivers/infiniband/sw/rxe/rxe.c
+++ b/drivers/infiniband/sw/rxe/rxe.c
@@ -310,6 +310,12 @@ static int rxe_newlink(const char *ibdev_name, struct net_device *ndev)
struct rxe_dev *exists;
int err = 0;
+ if (is_vlan_dev(ndev)) {
+ pr_err("rxe creation allowed on top of a real device only\n");
+ err = -EPERM;
+ goto err;
+ }
+
exists = rxe_get_dev_from_net(ndev);
if (exists) {
ib_device_put(&exists->ib_dev);
diff --git a/drivers/infiniband/sw/rxe/rxe_sysfs.c b/drivers/infiniband/sw/rxe/rxe_sysfs.c
index ccda5f5a3bc0..0a083c3d900a 100644
--- a/drivers/infiniband/sw/rxe/rxe_sysfs.c
+++ b/drivers/infiniband/sw/rxe/rxe_sysfs.c
@@ -73,6 +73,12 @@ static int rxe_param_set_add(const char *val, const struct kernel_param *kp)
return -EINVAL;
}
+ if (is_vlan_dev(ndev)) {
+ pr_err("rxe creation allowed on top of a real device only\n");
+ err = -EPERM;
+ goto err;
+ }
+
exists = rxe_get_dev_from_net(ndev);
if (exists) {
ib_device_put(&exists->ib_dev);
--
2.26.2
