On 1/16/12 6:33 PM, Joe Landman wrote: > On 01/16/2012 06:29 PM, Derek Yarnell wrote: >> Hi, >> >> So I wanted to test a gluster install w/ RDMA only support. RDMA is >> working with a successful running of ib_write_bw test between both >> nodes. After I start the gluster daemons I can no longer run the >> ib_write_bw tests and also gluster is showing errors on startup, >> > > 1st question: Did you upgrade from earlier 3.x? > > 2nd question: what does your mount command look like? > > For 3.2.x (x>2 or so), you need to mount > > mount -o intr -t glusterfs server:/volume.rdma /mnt > > for a volume. Note the ".rdma". Once you have this, make sure you are > correctly mounted. Do a df -h and see if it gives you a transport > disconnected. > > > Hi, 1) No this is just a new testbed. 2) I am not even to the mount stage yet. The errors in the log I am seeing are when I start the management daemon. I went through the process of creating the volume but still no dice. gluster peer probe 192.168.1.101 gluster volume create gtest replica 2 transport rdma 192.168.1.100:/scratch0/gluster 192.168.1.101:/scratch0/gluster gluster volume start gtest But the volume is not starting correctly, # gluster volume status gtest Brick status for volume: gtest Brick Port Online PID --------------------------------------------------------------------------- 192.168.1.100:/scratch0/gluster 24009 N 0 --------------------------------------------------------------------------- 192.168.1.101:/scratch0/gluster 24009 N 0 # cat bricks/scratch0-gluster.log [2012-01-16 19:17:41.928419] W [socket.c:420:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 [2012-01-16 19:17:41.928485] W [socket.c:1875:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported [2012-01-16 19:17:41.942922] I [graph.c:268:gf_add_cmdline_options] 0-gtest-server: adding option 'listen-port' for volume 'gtest-server' with value '24009' [2012-01-16 19:17:41.943651] W [options.c:647:xl_opt_validate] 0-gtest-server: option 'listen-port' is deprecated, preferred is 'transport.rdma.listen-port', continuing with correction [2012-01-16 19:17:41.946893] E [rdma.c:198:rdma_new_post] 0-rpc-transport/rdma: memory registration failed [2012-01-16 19:17:41.946932] E [rdma.c:2341:__rdma_create_posts] 0-rpc-transport/rdma: rdma.gtest-server: post creation failed [2012-01-16 19:17:41.948827] E [rdma.c:3861:rdma_get_device] 0-rpc-transport/rdma: rdma.gtest-server: could not allocate posts [2012-01-16 19:17:41.948848] E [rdma.c:3984:rdma_init] 0-rpc-transport/rdma: could not create rdma device for ipath0 [2012-01-16 19:17:41.948857] E [rdma.c:4806:init] 0-rdma.gtest-server: Failed to initialize IB Device [2012-01-16 19:17:41.948867] E [rpc-transport.c:325:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2012-01-16 19:17:41.948874] W [rpcsvc.c:1320:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2012-01-16 19:17:41.948882] W [server.c:659:init] 0-gtest-server: creation of listener failed [2012-01-16 19:17:41.948889] E [xlator.c:393:xlator_init] 0-gtest-server: Initialization of volume 'gtest-server' failed, review your volfile again [2012-01-16 19:17:41.948895] E [graph.c:325:glusterfs_graph_init] 0-gtest-server: initializing translator failed [2012-01-16 19:17:41.948901] E [graph.c:521:glusterfs_graph_activate] 0-graph: init failed [2012-01-16 19:17:41.949209] W [glusterfsd.c:750:cleanup_and_exit] (-->/opt/glusterfs/3.3beta2/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa2) [0x2b31e8bd3f02] (-->/opt/glusterfs/3.3beta2/sbin/glusterfsd(mgmt_getspec_cbk+0xc5) [0x408365] (-->/opt/glusterfs/3.3beta2/sbin/glusterfsd(glusterfs_process_volfp+0x17a) [0x40469a]))) 0-: received signum (0), shutting down -- --- Derek T. Yarnell University of Maryland Institute for Advanced Computer Studies
