Gluster version 3.7.6
I get "Connection failed. Please check if gluster daemon is operational.", sometimes. Again when I tried to create volume, the connection failed message appears again.
I am at loss to understand why this rdma.so is missing. Is this this necessary? How to get away from this error ?
Attached the log
Best regards
JK
[2016-07-25 09:11:33.016116] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/local/sbin/glusterd: Started running /usr/local/sbin/glusterd version 3.7.6 (args: /usr/local/sbin/glusterd -p /var/run/glusterd.pid) [2016-07-25 09:11:33.018409] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536 [2016-07-25 09:11:33.018444] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /var/lib/glusterd as working directory [2016-07-25 09:11:33.023273] E [rpc-transport.c:292:rpc_transport_load] 0-rpc-transport: /usr/lib/i386-linux-gnu/glusterfs/3.7.6/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2016-07-25 09:11:33.023296] W [rpc-transport.c:296:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2016-07-25 09:11:33.023310] W [rpcsvc.c:1597:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-07-25 09:11:33.023326] E [MSGID: 106243] [glusterd.c:1623:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport [2016-07-25 09:11:35.173226] I [MSGID: 106513] [glusterd-store.c:2069:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30706 pending frames: patchset: git://git.gluster.com/glusterfs.git signal received: 6 time of crash: 2016-07-25 09:11:35 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.7.6 /usr/lib/i386-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xf1)[0xb76150fb] /usr/lib/i386-linux-gnu/libglusterfs.so.0(gf_print_trace+0x253)[0xb762f478] /usr/local/sbin/glusterd(glusterfsd_print_trace+0x1a)[0x8050a29] [0xb76e7c8c] [0xb76e7cb0] /lib/i386-linux-gnu/libc.so.6(gsignal+0x47)[0xb741a607] /lib/i386-linux-gnu/libc.so.6(abort+0x143)[0xb741da33] /lib/i386-linux-gnu/libc.so.6(+0x27757)[0xb7413757] /lib/i386-linux-gnu/libc.so.6(+0x27807)[0xb7413807] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_store_update_volinfo+0x13b4)[0xb4288a6c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_store_retrieve_volume+0x138)[0xb4288da7] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_store_retrieve_volumes+0x31f)[0xb42897be] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_restore+0x3d)[0xb428e29b] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(init+0x1cc7)[0xb4210348] /usr/lib/i386-linux-gnu/libglusterfs.so.0(+0x1e4af)[0xb76114af] /usr/lib/i386-linux-gnu/libglusterfs.so.0(xlator_init+0x11b)[0xb76115e0] /usr/lib/i386-linux-gnu/libglusterfs.so.0(glusterfs_graph_init+0x39)[0xb7664e71] /usr/lib/i386-linux-gnu/libglusterfs.so.0(glusterfs_graph_activate+0xa5)[0xb7665951] /usr/local/sbin/glusterd(glusterfs_process_volfp+0x14f)[0x8051043] /usr/local/sbin/glusterd(glusterfs_volumes_init+0xe6)[0x805118d] /usr/local/sbin/glusterd(main+0x3c6)[0x8051574] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb7405a83] /usr/local/sbin/glusterd[0x804bb11] --------- [2016-07-25 09:13:02.262340] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/local/sbin/glusterd: Started running /usr/local/sbin/glusterd version 3.7.6 (args: /usr/local/sbin/glusterd -p /var/run/glusterd.pid) [2016-07-25 09:13:02.264362] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536 [2016-07-25 09:13:02.264394] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /var/lib/glusterd as working directory [2016-07-25 09:13:02.269187] E [rpc-transport.c:292:rpc_transport_load] 0-rpc-transport: /usr/lib/i386-linux-gnu/glusterfs/3.7.6/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2016-07-25 09:13:02.269208] W [rpc-transport.c:296:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2016-07-25 09:13:02.269223] W [rpcsvc.c:1597:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-07-25 09:13:02.269248] E [MSGID: 106243] [glusterd.c:1623:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport [2016-07-25 09:13:04.365966] I [MSGID: 106513] [glusterd-store.c:2069:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30706 [2016-07-25 09:13:04.366115] I [MSGID: 106194] [glusterd-store.c:3539:glusterd_store_retrieve_missed_snaps_list] 0-management: No missed snaps list. Final graph: +------------------------------------------------------------------------------+ [2016-07-25 09:13:04.383193] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option rpc-auth-allow-insecure on 7: option transport.socket.listen-backlog 128 8: option ping-timeout 30 9: option transport.socket.read-fail-log off 10: option transport.socket.keepalive-interval 2 11: option transport.socket.keepalive-time 10 12: option transport-type rdma 13: option working-directory /var/lib/glusterd 14: end-volume 15: +------------------------------------------------------------------------------+ [2016-07-25 09:13:04.384657] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-07-25 09:13:04.386137] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-07-25 09:13:04.708608] I [MSGID: 106143] [glusterd-pmap.c:229:pmap_registry_bind] 0-pmap: adding brick /home/jk/gluster/brick3 on port 49154 [2016-07-25 09:13:20.406364] E [glusterd-volume-ops.c:289:__glusterd_handle_create_volume] 0-management: JKMM: logging mgmt xlator [2016-07-25 09:13:20.406563] I [MSGID: 106544] [glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID: 871524ed-5401-482f-8f6c-d633ecb675ff [2016-07-25 09:13:20.411021] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick1 is already part of a volume [No data available] [2016-07-25 09:13:20.415655] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick2 is already part of a volume [No data available] [2016-07-25 09:13:20.420725] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick3 is already part of a volume [No data available] [2016-07-25 09:13:20.425370] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick4 is already part of a volume [No data available] [2016-07-25 09:13:20.430044] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick5 is already part of a volume [No data available] [2016-07-25 09:13:20.434656] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick6 is already part of a volume [No data available] [2016-07-25 09:13:20.439386] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick7 is already part of a volume [No data available] [2016-07-25 09:13:20.443916] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick8 is already part of a volume [No data available] pending frames: frame : type(0) op(0) patchset: git://git.gluster.com/glusterfs.git signal received: 8 time of crash: 2016-07-25 09:13:21 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.7.6 /usr/lib/i386-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xf1)[0xb76b20fb] /usr/lib/i386-linux-gnu/libglusterfs.so.0(gf_print_trace+0x253)[0xb76cc478] /usr/local/sbin/glusterd(glusterfsd_print_trace+0x1a)[0x8050a29] [0xb7784c8c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6c8f)[0xb433cc8f] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6d9c)[0xb433cd9c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb7a21)[0xb433da21] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8062)[0xb433e062] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8c25)[0xb433ec25] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb163a)[0xb433763a] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb952c)[0xb433f52c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xbca3e)[0xb4342a3e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(generate_client_volfiles+0x273)[0xb4342d65] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles+0xb1)[0xb4344a08] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles_and_notify_services+0x35)[0xb4344b23] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_create_volume+0x150c)[0xb437be80] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_commit_perform+0x6a)[0xb42e7fd9] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_commit_op_phase+0x1af)[0xb438ec9e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_sync_task_begin+0x8f1)[0xb43904ff] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_begin_synctask+0xb1)[0xb43907bf] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(__glusterd_handle_create_volume+0xc67)[0xb4374ef2] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_big_locked_handler+0x44)[0xb42c289b] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_handle_create_volume+0x27)[0xb4375006] /usr/lib/i386-linux-gnu/libglusterfs.so.0(synctask_wrap+0x33)[0xb770481e] /lib/i386-linux-gnu/libc.so.6(makecontext+0x4b)[0xb74c9c6b] [0xb2b05b48] [0x85e6400] --------- [2016-07-25 09:14:49.907596] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/local/sbin/glusterd: Started running /usr/local/sbin/glusterd version 3.7.6 (args: /usr/local/sbin/glusterd -p /var/run/glusterd.pid) [2016-07-25 09:14:49.910004] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536 [2016-07-25 09:14:49.910043] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /var/lib/glusterd as working directory [2016-07-25 09:14:49.915401] E [rpc-transport.c:292:rpc_transport_load] 0-rpc-transport: /usr/lib/i386-linux-gnu/glusterfs/3.7.6/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2016-07-25 09:14:49.915429] W [rpc-transport.c:296:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2016-07-25 09:14:49.915444] W [rpcsvc.c:1597:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-07-25 09:14:49.915460] E [MSGID: 106243] [glusterd.c:1623:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport [2016-07-25 09:14:52.008220] I [MSGID: 106513] [glusterd-store.c:2069:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30706 [2016-07-25 09:14:52.008365] I [MSGID: 106194] [glusterd-store.c:3539:glusterd_store_retrieve_missed_snaps_list] 0-management: No missed snaps list. Final graph: +------------------------------------------------------------------------------+ [2016-07-25 09:14:52.025330] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on 5: option rpc-auth.auth-null on 6: option rpc-auth-allow-insecure on 7: option transport.socket.listen-backlog 128 8: option ping-timeout 30 9: option transport.socket.read-fail-log off 10: option transport.socket.keepalive-interval 2 11: option transport.socket.keepalive-time 10 12: option transport-type rdma 13: option working-directory /var/lib/glusterd 14: end-volume 15: +------------------------------------------------------------------------------+ [2016-07-25 09:14:52.028681] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-07-25 09:14:52.028919] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-07-25 09:14:52.445760] I [MSGID: 106143] [glusterd-pmap.c:229:pmap_registry_bind] 0-pmap: adding brick /home/jk/gluster/brick3 on port 49154 [2016-07-25 09:14:59.734179] E [glusterd-volume-ops.c:289:__glusterd_handle_create_volume] 0-management: JKMM: logging mgmt xlator [2016-07-25 09:14:59.734373] I [MSGID: 106544] [glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID: 871524ed-5401-482f-8f6c-d633ecb675ff [2016-07-25 09:14:59.739041] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick1 is already part of a volume [No data available] [2016-07-25 09:14:59.743716] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick2 is already part of a volume [No data available] [2016-07-25 09:14:59.749343] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick3 is already part of a volume [No data available] [2016-07-25 09:14:59.754105] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick4 is already part of a volume [No data available] [2016-07-25 09:14:59.758601] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick5 is already part of a volume [No data available] [2016-07-25 09:14:59.763450] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick6 is already part of a volume [No data available] [2016-07-25 09:14:59.768193] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick7 is already part of a volume [No data available] [2016-07-25 09:14:59.772626] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick8 is already part of a volume [No data available] pending frames: frame : type(0) op(0) patchset: git://git.gluster.com/glusterfs.git signal received: 8 time of crash: 2016-07-25 09:15:00 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.7.6 /usr/lib/i386-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xf1)[0xb76d70fb] /usr/lib/i386-linux-gnu/libglusterfs.so.0(gf_print_trace+0x253)[0xb76f1478] /usr/local/sbin/glusterd(glusterfsd_print_trace+0x1a)[0x8050a29] [0xb77a9c8c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6c8f)[0xb4361c8f] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6d9c)[0xb4361d9c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb7a21)[0xb4362a21] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8062)[0xb4363062] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8c25)[0xb4363c25] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb163a)[0xb435c63a] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb952c)[0xb436452c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xbca3e)[0xb4367a3e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(generate_client_volfiles+0x273)[0xb4367d65] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles+0xb1)[0xb4369a08] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles_and_notify_services+0x35)[0xb4369b23] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_create_volume+0x150c)[0xb43a0e80] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_commit_perform+0x6a)[0xb430cfd9] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_commit_op_phase+0x1af)[0xb43b3c9e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_sync_task_begin+0x8f1)[0xb43b54ff] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_begin_synctask+0xb1)[0xb43b57bf] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(__glusterd_handle_create_volume+0xc67)[0xb4399ef2] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_big_locked_handler+0x44)[0xb42e789b] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_handle_create_volume+0x27)[0xb439a006] /usr/lib/i386-linux-gnu/libglusterfs.so.0(synctask_wrap+0x33)[0xb772981e] /lib/i386-linux-gnu/libc.so.6(makecontext+0x4b)[0xb74eec6b] [0x9d9ba68] [0xbf8d1d00] --------- [2016-07-25 09:27:34.531920] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/local/sbin/glusterd: Started running /usr/local/sbin/glusterd version 3.7.6 (args: /usr/local/sbin/glusterd -p /var/run/glusterd.pid) [2016-07-25 09:27:34.534310] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536 [2016-07-25 09:27:34.534349] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /var/lib/glusterd as working directory [2016-07-25 09:27:34.539949] E [rpc-transport.c:292:rpc_transport_load] 0-rpc-transport: /usr/lib/i386-linux-gnu/glusterfs/3.7.6/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2016-07-25 09:27:34.539979] W [rpc-transport.c:296:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2016-07-25 09:27:34.539997] W [rpcsvc.c:1597:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-07-25 09:27:34.540015] E [MSGID: 106243] [glusterd.c:1623:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport [2016-07-25 09:27:36.633564] I [MSGID: 106513] [glusterd-store.c:2069:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30706 [2016-07-25 09:27:36.633701] I [MSGID: 106194] [glusterd-store.c:3539:glusterd_store_retrieve_missed_snaps_list] 0-management: No missed snaps list. Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on [2016-07-25 09:27:36.651528] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 5: option rpc-auth.auth-null on 6: option rpc-auth-allow-insecure on 7: option transport.socket.listen-backlog 128 8: option ping-timeout 30 9: option transport.socket.read-fail-log off 10: option transport.socket.keepalive-interval 2 11: option transport.socket.keepalive-time 10 12: option transport-type rdma 13: option working-directory /var/lib/glusterd 14: end-volume 15: +------------------------------------------------------------------------------+ [2016-07-25 09:27:36.653001] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-07-25 09:27:36.654040] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-07-25 09:27:36.975746] I [MSGID: 106143] [glusterd-pmap.c:229:pmap_registry_bind] 0-pmap: adding brick /home/jk/gluster/brick3 on port 49154 [2016-07-25 09:28:09.846238] I [MSGID: 106488] [glusterd-handler.c:1498:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2016-07-25 09:29:14.967117] E [glusterd-volume-ops.c:289:__glusterd_handle_create_volume] 0-management: JKMM: logging mgmt xlator [2016-07-25 09:29:14.967315] I [MSGID: 106544] [glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID: 871524ed-5401-482f-8f6c-d633ecb675ff [2016-07-25 09:29:14.971812] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/brick1 is already part of a volume [File exists] [2016-07-25 09:29:14.976463] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/brick2 is already part of a volume [File exists] [2016-07-25 09:29:14.981918] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/brick3 is already part of a volume [File exists] [2016-07-25 09:29:14.986633] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/brick4 is already part of a volume [File exists] [2016-07-25 09:29:14.991247] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/brick5 is already part of a volume [File exists] [2016-07-25 09:29:14.996074] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/brick6 is already part of a volume [File exists] [2016-07-25 09:30:17.546438] E [glusterd-volume-ops.c:289:__glusterd_handle_create_volume] 0-management: JKMM: logging mgmt xlator [2016-07-25 09:30:17.550577] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick1 is already part of a volume [No data available] [2016-07-25 09:30:17.555291] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick2 is already part of a volume [No data available] [2016-07-25 09:30:17.560261] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick3 is already part of a volume [No data available] [2016-07-25 09:30:17.565266] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick4 is already part of a volume [No data available] [2016-07-25 09:30:17.570035] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick5 is already part of a volume [No data available] [2016-07-25 09:30:17.575144] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick6 is already part of a volume [No data available] [2016-07-25 09:30:17.580504] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick7 is already part of a volume [No data available] [2016-07-25 09:30:17.585571] E [MSGID: 106451] [glusterd-utils.c:5800:glusterd_is_path_in_use] 0-management: /home/jk/gluster/dsi-brick8 is already part of a volume [No data available] pending frames: frame : type(0) op(0) patchset: git://git.gluster.com/glusterfs.git signal received: 8 time of crash: 2016-07-25 09:30:18 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.7.6 /usr/lib/i386-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xf1)[0xb769e0fb] /usr/lib/i386-linux-gnu/libglusterfs.so.0(gf_print_trace+0x253)[0xb76b8478] /usr/local/sbin/glusterd(glusterfsd_print_trace+0x1a)[0x8050a29] [0xb7770c8c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6c8f)[0xb4328c8f] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6d9c)[0xb4328d9c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb7a21)[0xb4329a21] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8062)[0xb432a062] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8c25)[0xb432ac25] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb163a)[0xb432363a] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb952c)[0xb432b52c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xbca3e)[0xb432ea3e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(generate_client_volfiles+0x273)[0xb432ed65] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles+0xb1)[0xb4330a08] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles_and_notify_services+0x35)[0xb4330b23] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_create_volume+0x150c)[0xb4367e80] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_commit_perform+0x6a)[0xb42d3fd9] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_commit_op_phase+0x1af)[0xb437ac9e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_sync_task_begin+0x8f1)[0xb437c4ff] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_begin_synctask+0xb1)[0xb437c7bf] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(__glusterd_handle_create_volume+0xc67)[0xb4360ef2] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_big_locked_handler+0x44)[0xb42ae89b] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_handle_create_volume+0x27)[0xb4361006] /usr/lib/i386-linux-gnu/libglusterfs.so.0(synctask_wrap+0x33)[0xb76f081e] /lib/i386-linux-gnu/libc.so.6(makecontext+0x4b)[0xb74b5c6b] [0x9f8ac30] [0x69318e00] --------- [2016-07-25 09:33:19.204603] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/local/sbin/glusterd: Started running /usr/local/sbin/glusterd version 3.7.6 (args: /usr/local/sbin/glusterd -p /var/run/glusterd.pid) [2016-07-25 09:33:19.206775] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536 [2016-07-25 09:33:19.206809] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /var/lib/glusterd as working directory [2016-07-25 09:33:19.211670] E [rpc-transport.c:292:rpc_transport_load] 0-rpc-transport: /usr/lib/i386-linux-gnu/glusterfs/3.7.6/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2016-07-25 09:33:19.211697] W [rpc-transport.c:296:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2016-07-25 09:33:19.211713] W [rpcsvc.c:1597:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2016-07-25 09:33:19.211728] E [MSGID: 106243] [glusterd.c:1623:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport [2016-07-25 09:33:21.309475] I [MSGID: 106513] [glusterd-store.c:2069:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 30706 [2016-07-25 09:33:21.309620] I [MSGID: 106194] [glusterd-store.c:3539:glusterd_store_retrieve_missed_snaps_list] 0-management: No missed snaps list. Final graph: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option rpc-auth.auth-glusterfs on 4: option rpc-auth.auth-unix on [2016-07-25 09:33:21.327302] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 5: option rpc-auth.auth-null on 6: option rpc-auth-allow-insecure on 7: option transport.socket.listen-backlog 128 8: option ping-timeout 30 9: option transport.socket.read-fail-log off 10: option transport.socket.keepalive-interval 2 11: option transport.socket.keepalive-time 10 12: option transport-type rdma 13: option working-directory /var/lib/glusterd 14: end-volume 15: +------------------------------------------------------------------------------+ [2016-07-25 09:33:21.330092] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-07-25 09:33:21.330250] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-07-25 09:33:21.647687] I [MSGID: 106143] [glusterd-pmap.c:229:pmap_registry_bind] 0-pmap: adding brick /home/jk/gluster/brick3 on port 49154 [2016-07-25 09:33:31.480189] E [glusterd-volume-ops.c:289:__glusterd_handle_create_volume] 0-management: JKMM: logging mgmt xlator [2016-07-25 09:33:31.480415] I [MSGID: 106544] [glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID: 871524ed-5401-482f-8f6c-d633ecb675ff pending frames: frame : type(0) op(0) patchset: git://git.gluster.com/glusterfs.git signal received: 8 time of crash: 2016-07-25 09:33:32 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.7.6 /usr/lib/i386-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xf1)[0xb76190fb] /usr/lib/i386-linux-gnu/libglusterfs.so.0(gf_print_trace+0x253)[0xb7633478] /usr/local/sbin/glusterd(glusterfsd_print_trace+0x1a)[0x8050a29] [0xb76ebc8c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6c8f)[0xb42a3c8f] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb6d9c)[0xb42a3d9c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb7a21)[0xb42a4a21] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8062)[0xb42a5062] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb8c25)[0xb42a5c25] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb163a)[0xb429e63a] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xb952c)[0xb42a652c] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(+0xbca3e)[0xb42a9a3e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(generate_client_volfiles+0x273)[0xb42a9d65] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles+0xb1)[0xb42aba08] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_create_volfiles_and_notify_services+0x35)[0xb42abb23] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_create_volume+0x150c)[0xb42e2e80] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_commit_perform+0x6a)[0xb424efd9] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_commit_op_phase+0x1af)[0xb42f5c9e] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(gd_sync_task_begin+0x8f1)[0xb42f74ff] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_op_begin_synctask+0xb1)[0xb42f77bf] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(__glusterd_handle_create_volume+0xc67)[0xb42dbef2] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_big_locked_handler+0x44)[0xb422989b] /usr/local/lib/glusterfs/3.7.6/xlator/mgmt/glusterd.so(glusterd_handle_create_volume+0x27)[0xb42dc006] /usr/lib/i386-linux-gnu/libglusterfs.so.0(synctask_wrap+0x33)[0xb766b81e] /lib/i386-linux-gnu/libc.so.6(makecontext+0x4b)[0xb7430c6b] [0x8d10bf8] /usr/lib/i386-linux-gnu/libglusterfs.so.0(synctask_new1+0x4e)[0xb766beec] /usr/lib/i386-linux-gnu/libglusterfs.so.0(synctask_new+0x41)[0xb766bf5a] /usr/lib/i386-linux-gnu/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x182)[0xb75e38d2] ---------
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
