Hi, So I wanted to test a gluster install w/ RDMA only support. RDMA is working with a successful running of ib_write_bw test between both nodes. After I start the gluster daemons I can no longer run the ib_write_bw tests and also gluster is showing errors on startup, [2012-01-16 18:27:58.199062] I [graph.c:268:gf_add_cmdline_options] 0-management: adding option 'upgrade' for volume 'management' with value 'on' [2012-01-16 18:27:58.199105] I [glusterd.c:574:init] 0-management: Using /etc/glusterd as working directory [2012-01-16 18:27:58.199707] E [rpc-transport.c:261:rpc_transport_load] 0-rpc-transport: /opt/glusterfs/3.3beta2/lib64/glusterfs/3.3beta2/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2012-01-16 18:27:58.199729] E [rpc-transport.c:265:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2012-01-16 18:27:58.199736] W [rpcsvc.c:1320:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2012-01-16 18:27:58.199788] I [glusterd.c:89:glusterd_uuid_init] 0-glusterd: retrieved UUID: e76627a1-4d1b-4d96-beef-ad5811970faf [2012-01-16 18:27:58.200437] I [glusterd.c:294:glusterd_check_gsync_present] 0-: geo-replication module not installed in the system Given volfile: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option working-directory /etc/glusterd 4: option transport-type socket,rdma 5: option transport.socket.keepalive-time 10 6: option transport.socket.keepalive-interval 2 7: option transport.socket.read-fail-log off 8: end-volume +------------------------------------------------------------------------------+ [2012-01-16 18:28:08.202896] W [glusterfsd.c:750:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3020ad44bd] (-->/lib64/libpthread.so.0 [0x302160673d] (-->glusterd(glusterfs_sigwaiter+0x17c) [0x404a0c]))) 0-: received signum (15), shutting down [2012-01-16 18:28:18.248888] I [glusterd.c:574:init] 0-management: Using /etc/glusterd as working directory [2012-01-16 18:28:18.271281] E [rdma.c:198:rdma_new_post] 0-rpc-transport/rdma: memory registration failed [2012-01-16 18:28:18.271329] E [rdma.c:2341:__rdma_create_posts] 0-rpc-transport/rdma: rdma.management: post creation failed [2012-01-16 18:28:18.273491] E [rdma.c:3861:rdma_get_device] 0-rpc-transport/rdma: rdma.management: could not allocate posts [2012-01-16 18:28:18.273512] E [rdma.c:3984:rdma_init] 0-rpc-transport/rdma: could not create rdma device for ipath0 [2012-01-16 18:28:18.273521] E [rdma.c:4806:init] 0-rdma.management: Failed to initialize IB Device [2012-01-16 18:28:18.273531] E [rpc-transport.c:325:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed [2012-01-16 18:28:18.273542] W [rpcsvc.c:1320:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2012-01-16 18:28:18.273621] I [glusterd.c:89:glusterd_uuid_init] 0-glusterd: retrieved UUID: e76627a1-4d1b-4d96-beef-ad5811970faf [2012-01-16 18:28:18.275417] I [glusterd.c:294:glusterd_check_gsync_present] 0-: geo-replication module not installed in the system Given volfile: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option working-directory /etc/glusterd 4: option transport-type socket,rdma 5: option transport.socket.keepalive-time 10 6: option transport.socket.keepalive-interval 2 7: option transport.socket.read-fail-log off 8: end-volume +------------------------------------------------------------------------------+ Any pointers? I have tried both 3.2.5 and 3.3b2. Thanks, derek -- --- Derek T. Yarnell University of Maryland Institute for Advanced Computer Studies
