Hi, it happens that RDMA gluster transport does not works anymore after I have configured ibscif virtual connector for Infiniband in a server with a XeonPHI coprocessor. I have CentOS 6.6 and GLUSTER 3.8.5, OFED 3.12-1 MPSS 3.5.2 and I have followed the installation instructions of MPSS_Users_Guide (Revision 3.5) that suggested to remove compat-rdma-devel and compat-rdma packages. I have noticed that running the command: ib_send_bw gives the following error: # ib_send_bw ************************************ * Waiting for client to connect... * ************************************ --------------------------------------------------------------------- ------------------ Send BW Test Dual-port : OFF Device : scif0 Number of qps : 1 Transport type : IW Connection type : RC Using SRQ : OFF RX depth : 512 CQ Moderation : 100 Mtu : 2048[B] Link type : Ethernet Gid index : 0 Max inline data : 0[B] rdma_cm QPs : OFF Data ex. method : Ethernet --------------------------------------------------------------------- ------------------ local address: LID 0x3e8 QPN 0x0003 PSN 0x123123 GID: 76:121:186:102:03:119:00:00:00:00:00:00:00:00:00:00 ethernet_read_keys: Couldn't read remote address Unable to read to socket/rdam_cm Failed to exchange data between server and clients Instead using the output of the command ib_send_bw -d qib0 gives correct results: # ib_send_bw -d qib0 ************************************ * Waiting for client to connect... * ************************************ --------------------------------------------------------------------- ------------------ Send BW Test Dual-port : OFF Device : qib0 Number of qps : 1 Transport type : IB Connection type : RC Using SRQ : OFF RX depth : 512 CQ Moderation : 100 Mtu : 2048[B] Link type : IB Max inline data : 0[B] rdma_cm QPs : OFF Data ex. method : Ethernet --------------------------------------------------------------------- ------------------ local address: LID 0x0a QPN 0x0169 PSN 0xe0b768 remote address: LID 0x20 QPN 0x28b280 PSN 0xc3008c --------------------------------------------------------------------- ------------------ #bytes #iterations BW peak[MB/sec] BW average[MB/sec] MsgRate[Mpps] 65536 1000 0.00 2160.87 0.034 574 --------------------------------------------------------------------- ------------------ _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users
