On Thu, Jan 20, 2022 at 12:32:12PM +0100, Alexander Kalentyev wrote: > Dear Robert, > > Thank you for your help! > What I did was : > ipv6calc --in prefix+mac fe80:: XX:XX:XX:XX:XX:XX > (with XX:XX:XX:XX:XX:XX one have to insert the MAC address of the NIC) > And then > udo ip addr add dev <dev> fe80::ZZZZ:ZZZZ:ZZZZ:ZZZZ/64 > (with ZZZ:ZZZZ:ZZZZ the address genereted by ipv6calc) > sudo rdma link add rxe0 type rxe netdev <dev> > After that everything is working as expected: > >ibv_rc_pingpong -d rxe0 -g 0 192.168.0.176 > local address: LID 0x0000, QPN 0x000012, PSN 0xf85048, GID > ZZZZ::ZZZZ:ZZZZ:ZZZZ:ZZZZ > remote address: LID 0x0000, QPN 0x000011, PSN 0xf7c7b7, GID > ZZZZ::ZZZZ:ZZZZ:ZZZZ:ZZZZ > 8192000 bytes in 0.01 seconds = 4743.49 Mbit/sec > 1000 iters in 0.01 seconds = 13.82 usec/iter > > So, I think it make sense to include this information in a README on > github. Otherwise somebody like me can spend a week trying to figure > out what is going on! PR is more than welcomed. Thanks > > Thanks once again! > Regards, > Alexander Kalentev > > ср, 19 янв. 2022 г. в 20:54, Pearson, Robert B <robert.pearson2@xxxxxxx>: > > > > > > > > -----Original Message----- > > From: Alexander Kalentyev <comrad.karlovich@xxxxxxxxx> > > Sent: Wednesday, January 19, 2022 11:53 AM > > > > > > Anyway the ibv_rc_pingpong shows an error: > > > > >ibv_rc_pingpong -d rxe0 -g 0 > > local address: LID 0x0000, QPN 0x000015, PSN 0x015dd8, GID > > fe80::4a51:c5ff:fef6:e159 > > Failed to modify QP to RTR > > Couldn't connect to remote QP > > > > > > > Alexander, > > > > I use a script to restart rxe after changing anything it looks like > > > > #!/bin/bash > > > > export LD_LIBRARY_PATH=<path to rdma-core>/rdma-core/build/lib:/usr/lib > > > > sudo rmmod rdma_rxe > > sudo modprobe rdma_rxe > > > > sudo ip link set dev enp0s3 mtu 4500 > > sudo ip addr add dev enp0s3 fe80::0a00:27ff:fe35:5ea7/64 > > sudo rdma link add rxe0 type rxe netdev enp0s3 > > > > The important line is adding the ipv6 address which corresponds with the MAC address of > > The ethernet nic which is > > > > 08:00:27:35:5e:a7 > > > > Some OSes (like mine) do not create this address automatically but mangle the address. > > But the rdma core driver seems to expect all roce providers to have it. > > > > Hope this helps. > > > > Bob Pearson > > rpearson@xxxxxxx
