Hi guys,
Is there any configuration (distro/kernel/rdma-core branch) you can
suggest to try out software rdma (rxe)?
I'm trying to get software rdma working to enable local testing on the
vm (vmware) without going proper hardware and so far can't get
pingpong tool working.
I've tried CentOS 7.2 first by building vanilla 4.9.5 with rxe enabled
Now I've tried Ubuntu 16.04.1, using built ubuntu 4.9.5 build with
rxe. I've noticed that rxe is actually enabled by default as module so
now tried just released 4.9.6 as well.
I've tried linux-rdma/rdma-core master and rdma-core-12 tag and lately
rleon/rdma-core debian-fixes-last
What I get is (in all the cases I've tried) that module is loaded and
looks ok, however serverside ibv_rc_pingpong fails when client
connects with:
root@xdev:~/rdma-core-rleon# ibv_rc_pingpong -d rxe0 -g 0 -i 1
local address: LID 0x0000, QPN 0x000012, PSN 0xaa7393, GID
fe80::20c:29ff:fe87:5082
remote address: LID 0x0000, QPN 0x000013, PSN 0xf70e00, GID
fe80::20c:29ff:fe87:5082
Completion for unknown wr_id 0
parse WC failed 2
Below is output of other checks, all looks good as far as I understand.
If you can suggest anything, please let me know.
Thanks!
Fedor
root@xdev:~# uname -a
Linux xdev 4.9.6-040906-generic #201701260330 SMP Thu Jan 26 08:32:10
UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
root@xdev:~# lsmod | grep rxe
rdma_rxe 114688 1
ip6_udp_tunnel 16384 1 rdma_rxe
udp_tunnel 16384 1 rdma_rxe
ib_core 204800 6 rdma_rxe,ib_cm,rdma_cm,ib_uverbs,iw_cm,rdma_ucm
root@xdev:~# lsmod | grep ib_uverbs
ib_uverbs 65536 4 rdma_ucm
ib_core 204800 6 rdma_rxe,ib_cm,rdma_cm,ib_uverbs,iw_cm,rdma_ucm
root@xdev:~# rxe_cfg status
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU
ens33 yes e1000 1500 192.168.237.132 rxe0 1024 (3)
root@xdev:~# ibv_devices
device node GUID
------ ----------------
rxe0 020c29fffe875082
root@xdev:~# ibv_devinfo -d rxe0
hca_id: rxe0
transport: InfiniBand (0)
fw_ver: 0.0.0
node_guid: 020c:29ff:fe87:5082
sys_image_guid: 0000:0000:0000:0000
vendor_id: 0x0000
vendor_part_id: 0
hw_ver: 0x0
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
link_layer: Ethernet
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
