ibv_rc_pingpong failes on softroce device using GID[0]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi All
I am using ubuntu 21.10. I used the following command to create a
softroce device rxe0:
sudo rdma link add rxe0 type rxe netdev enp0s3

rxe0's GID[0] is not an real IP address assigned to my enp0s3. It
seems to be something softroce generated from the node_guid (seems to
be generated from mac address of enp0s3).
Is this expected behavior? How to use this GID given that it is not
reachable because it is not a real IP address?

I used ibv_rc_pingpong -d rxe0 -g 0 to setup a server with this GID,
the server failed with "Failed to modify QP to RTR" when client tries
to talk with it with client's GID[0] using command "./ibv_rc_pingpong
-d rxe0 -g 0 10.0.0.233". Is this because the GID[0] is not a real IP?

ibv_rc_pingpong works if i use any other GID, (eg. GID[1] - GID[5])

Here is the output of ibv_devinfo and ip addr. you can see GID[0]
fe80::a00:27ff:fe2e:f7a5 is not a real ip address assigned to enp0s3
(where rxe0 was created).

huiwang@huiwang-VirtualBox:~/rdma-core$ ibv_devinfo -v
hca_id: rxe0
        transport:                      InfiniBand (0)
        fw_ver:                         0.0.0
        node_guid:                      0a00:27ff:fe2e:f7a5
        sys_image_guid:                 0a00:27ff:fe2e:f7a5
....
                        GID[  0]:
fe80::a00:27ff:fe2e:f7a5, RoCE v2
                        GID[  1]:
fe80::9bd3:885:53df:9fc4, RoCE v2
                        GID[  2]:               ::ffff:10.0.0.151, RoCE v2
                        GID[  3]:
2601:647:5380:2210:19df:d855:54a9:cc9f, RoCE v2
                        GID[  4]:
2601:647:5380:2210:e3f3:7e57:504b:7e5c, RoCE v2
                        GID[  5]:
2601:647:5380:2210::3be1, RoCE v2

huiwang@huiwang-VirtualBox:~/rdma-core/build/bin$ ip addr
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
state UP group default qlen 1000
    link/ether 08:00:27:2e:f7:a5 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.151/64 brd 10.0.0.255 scope global dynamic noprefixroute enp0s3
       valid_lft 143273sec preferred_lft 143273sec
    inet6 2601:647:5380:2210::3be1/128 scope global dynamic noprefixroute
       valid_lft 575276sec preferred_lft 575276sec
    inet6 2601:647:5380:2210:e3f3:7e57:504b:7e5c/64 scope global
temporary dynamic
       valid_lft 255135sec preferred_lft 56812sec
    inet6 2601:647:5380:2210:19df:d855:54a9:cc9f/64 scope global
dynamic mngtmpaddr noprefixroute
       valid_lft 255135sec preferred_lft 255135sec
    inet6 fe80::9bd3:885:53df:9fc4/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

Thanks,
Hui



[Index of Archives]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Photo]     [Yosemite News]     [Yosemite Photos]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux