While the device is detected by ibv_devices and rping works, I can't get ibv_rc_pingpong working (and as far as I understand, RC should be supported by the iWARP driver) rping works: server$ rping -s -a 10.30.10.211 -v server ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr server ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs server DISCONNECT EVENT... wait for RDMA_READ_ADV state 10 client$ rping -c -a 10.30.10.211 -C 2 -v (output omitted) But ibv_rc_pingpong doesn't server$ ibv_rc_pingpong -d iwp____ local address: LID 0x0000, QPN 0x000001, PSN 0xb8aafc, GID :: Failed to modify QP to RTS Couldn't connect to remote QP client$ ibv_rc_pingpong -d iwp____ 10.30.10.211 local address: LID 0x0000, QPN 0x000001, PSN 0x71abc5, GID :: client read/write: Protocol not supported Couldn't read/write remote address dmesg says: [ 93.957733] iwpm_register_pid: Unable to send a nlmsg (client = 2) And OpenMPI/UCX doesn't want to use the transport either: [1566564168.338103] [host:23059:0] select.c:410 UCX ERROR no active messages transport to <no debug data>: mm/posix - Destination is unreachable, mm/sysv - Destination is unreachable, self/self - Destination is unreachable, rdmacm/sockaddr - no am bcopy, cma/cma - no am bcopy Is this a bug or a limitation of siw? Thanks, Marcin On 22.08.2019 18:24, Marcin Mielniczuk wrote: > Thanks a lot, this did the trick. I think this is worth documenting > somewhere. > Would README.md in the rdma-core repo be a good place? > > Why > > On 22.08.2019 17:52, Leon Romanovsky wrote: >> On Thu, Aug 22, 2019 at 09:13:25PM +0530, Krishnamraju Eraparaju wrote: >>> On Thursday, August 08/22/19, 2019 at 17:08:49 +0200, Marcin Mielniczuk wrote: >>>> Hi, >>>> >>>> I'm trying to test the recently merged siw module. >>>> I'm running kernel 5.3-rc5 (taken from the Ubuntu mainline-kernel >>>> repository [1]) on Ubuntu 18.04 (bionic). >>>> I also manually installed rdma-core 25.0 from tarball, using the >>>> included Debian packaging. I installed all the packages but ibacm. >>>> >>>> After booting the new kernel I manually loaded the kernel module by >>>> >>>> modprobe siw >>>> modprobe rdma_ucm >>>> >>>> Then ibv_devinfo shows: "No IB devices found". >>>> dmesg only shows: >>>> [ 29.856751] SoftiWARP attached >>>> >>>> According to this tutorial, [2] it should be enough to just load the siw >>>> module. (unlike RXE, where one needs to use rxe_cfg to set up the >>>> interface) >>>> Is this a bug in siw or just a configuration issue on my side? >>> Have you done "rdma link"? >>> >>> rdma link add <NAME> type siw netdev <NETDEV> >>> >>> http://man7.org/linux/man-pages/man8/rdma-link.8.html >> BTW, the same goes for RXE and rxe_cfg is discouraged. >> >> Thanks >> >>>> Thanks, >>>> Marcin >>>> >>>> [1] https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.3-rc5/ >>>> [2] https://budevg.github.io/posts/tutorials/2017/04/29/rdma-101-1.html >>>> >
