Hi All
I am using ubuntu 21.10. I used the following command to create a
softroce device rxe0:
sudo rdma link add rxe0 type rxe netdev enp0s3
rxe0's GID[0] is not an real IP address assigned to my enp0s3. It
seems to be something softroce generated from the node_guid (seems to
be generated from mac address of enp0s3).
Is this expected behavior? How to use this GID given that it is not
reachable because it is not a real IP address?
I used ibv_rc_pingpong -d rxe0 -g 0 to setup a server with this GID,
the server failed with "Failed to modify QP to RTR" when client tries
to talk with it with client's GID[0] using command "./ibv_rc_pingpong
-d rxe0 -g 0 10.0.0.233". Is this because the GID[0] is not a real IP?
ibv_rc_pingpong works if i use any other GID, (eg. GID[1] - GID[5])
Here is the output of ibv_devinfo and ip addr. you can see GID[0]
fe80::a00:27ff:fe2e:f7a5 is not a real ip address assigned to enp0s3
(where rxe0 was created).
huiwang@huiwang-VirtualBox:~/rdma-core$ ibv_devinfo -v
hca_id: rxe0
transport: InfiniBand (0)
fw_ver: 0.0.0
node_guid: 0a00:27ff:fe2e:f7a5
sys_image_guid: 0a00:27ff:fe2e:f7a5
....
GID[ 0]:
fe80::a00:27ff:fe2e:f7a5, RoCE v2
GID[ 1]:
fe80::9bd3:885:53df:9fc4, RoCE v2
GID[ 2]: ::ffff:10.0.0.151, RoCE v2
GID[ 3]:
2601:647:5380:2210:19df:d855:54a9:cc9f, RoCE v2
GID[ 4]:
2601:647:5380:2210:e3f3:7e57:504b:7e5c, RoCE v2
GID[ 5]:
2601:647:5380:2210::3be1, RoCE v2
huiwang@huiwang-VirtualBox:~/rdma-core/build/bin$ ip addr
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
state UP group default qlen 1000
link/ether 08:00:27:2e:f7:a5 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.151/64 brd 10.0.0.255 scope global dynamic noprefixroute enp0s3
valid_lft 143273sec preferred_lft 143273sec
inet6 2601:647:5380:2210::3be1/128 scope global dynamic noprefixroute
valid_lft 575276sec preferred_lft 575276sec
inet6 2601:647:5380:2210:e3f3:7e57:504b:7e5c/64 scope global
temporary dynamic
valid_lft 255135sec preferred_lft 56812sec
inet6 2601:647:5380:2210:19df:d855:54a9:cc9f/64 scope global
dynamic mngtmpaddr noprefixroute
valid_lft 255135sec preferred_lft 255135sec
inet6 fe80::9bd3:885:53df:9fc4/64 scope link noprefixroute
valid_lft forever preferred_lft forever
Thanks,
Hui
