Can you verify that ibv_srq_pingpong works from the server where this log file is from? Thanks, Avati On Wed, Dec 1, 2010 at 7:44 PM, Jeremy Stout <stout.jeremy at gmail.com> wrote: > Whenever I try to start or mount a GlusterFS 3.1.1 volume that uses > RDMA, I'm seeing the following error messages in the log file on the > server: > [2010-11-30 18:37:53.51270] I [nfs.c:652:init] nfs: NFS service started > [2010-11-30 18:37:53.51362] W [dict.c:1204:data_to_str] dict: @data=(nil) > [2010-11-30 18:37:53.51375] W [dict.c:1204:data_to_str] dict: @data=(nil) > [2010-11-30 18:37:53.59628] E [rdma.c:2066:rdma_create_cq] > rpc-transport/rdma: testdir-client-0: creation of send_cq failed > [2010-11-30 18:37:53.59851] E [rdma.c:3771:rdma_get_device] > rpc-transport/rdma: testdir-client-0: could not create CQ > [2010-11-30 18:37:53.59925] E [rdma.c:3957:rdma_init] > rpc-transport/rdma: could not create rdma device for mthca0 > [2010-11-30 18:37:53.60009] E [rdma.c:4789:init] testdir-client-0: > Failed to initialize IB Device > [2010-11-30 18:37:53.60030] E [rpc-transport.c:971:rpc_transport_load] > rpc-transport: 'rdma' initialization failed > > On the client, I see: > [2010-11-30 18:43:49.653469] W [io-stats.c:1644:init] testdir: > dangling volume. check volfile > [2010-11-30 18:43:49.653573] W [dict.c:1204:data_to_str] dict: @data=(nil) > [2010-11-30 18:43:49.653607] W [dict.c:1204:data_to_str] dict: @data=(nil) > [2010-11-30 18:43:49.736275] E [rdma.c:2066:rdma_create_cq] > rpc-transport/rdma: testdir-client-0: creation of send_cq failed > [2010-11-30 18:43:49.736651] E [rdma.c:3771:rdma_get_device] > rpc-transport/rdma: testdir-client-0: could not create CQ > [2010-11-30 18:43:49.736689] E [rdma.c:3957:rdma_init] > rpc-transport/rdma: could not create rdma device for mthca0 > [2010-11-30 18:43:49.736805] E [rdma.c:4789:init] testdir-client-0: > Failed to initialize IB Device > [2010-11-30 18:43:49.736841] E > [rpc-transport.c:971:rpc_transport_load] rpc-transport: 'rdma' > initialization failed > > This results in an unsuccessful mount. > > I created the mount using the following commands: > /usr/local/glusterfs/3.1.1/sbin/gluster volume create testdir > transport rdma submit-1:/exports > /usr/local/glusterfs/3.1.1/sbin/gluster volume start testdir > > To mount the directory, I use: > mount -t glusterfs submit-1:/testdir /mnt/glusterfs > > I don't think it is an Infiniband problem since GlusterFS 3.0.6 and > GlusterFS 3.1.0 worked on the same systems. For GlusterFS 3.1.0, the > commands listed above produced no error messages. > > If anyone can provide help with debugging these error messages, it > would be appreciated. > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >
