Sadly no go with 3.1.4qa3: patchset: v3.1.4qa3 signal received: 11 time of crash: 2011-04-12 16:16:34 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.1.4qa3 /lib64/libc.so.6(+0x33140)[0x7f68d48c8140] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/xlator/mgmt/glusterd.so(+0x3323b)[0x7f68d311e23b] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/xlator/mgmt/glusterd.so(glusterd_op_sm+0xee)[0x7f68d311d6ce] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/xlator/mgmt/glusterd.so(glusterd_handle_cli_start_volume+0x8f)[0x7f68d310c91f] /opt/glusterfs/3.1.4qa3/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x28f)[0x7f68d526e01f] /opt/glusterfs/3.1.4qa3/lib64/libgfrpc.so.0(rpcsvc_notify+0x123)[0x7f68d526e653] /opt/glusterfs/3.1.4qa3/lib64/libgfrpc.so.0(rpc_transport_notify+0x2d)[0x7f68d527221d] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7f68d2ee3b44] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/rpc-transport/socket.so(socket_event_handler+0xb3)[0x7f68d2ee3de3] /opt/glusterfs/3.1.4qa3/lib64/libglusterfs.so.0(+0x37d42)[0x7f68d54b2d42] /opt/glusterfs/3.1.4qa3/sbin/glusterd(main+0x263)[0x4054b3] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7f68d48b3e5d] /opt/glusterfs/3.1.4qa3/sbin/glusterd[0x402db9] On 04/12/2011 04:07 PM, Mark "Naoki" Rogers wrote:
I had an interesting situation with a (3x3) distribute+replicate setup. One of the machines died (memory problem, hard lock) and gluster didn't exactly like it (although that's another story). Anyway I decided to keep on truckin' and use a 5 node distribute (since this is mainly a network test anyway) and the volume creation was ok but startup failed with the old "cli op: 16" message (note: would be nice if that actually said lock file issue if that's what it is - seems to be solved by a restart anyway). I restarted glusterd but everytime I try and "volume start" it just crashes immediately: [2011-04-12 15:36:39.861099] I [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: af4007ac-804c-433f-a6e9-0a7d6cafe432, host: pdbsearch12 [2011-04-12 15:36:39.861123] I [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: 5e4ef932-cca7-4440-a8e3-4415b84c2307, host: pdbsearch13 [2011-04-12 15:36:39.861148] I [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: df6c1fb1-cf5d-4a6b-ba15-fb26319e37db, host: pdbsearch14 [2011-04-12 15:36:39.861172] I [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: 2cc51957-2c92-4d24-a53c-c7ded19e80b7, host: pdbsearch15 [2011-04-12 15:36:39.861695] I [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: Received ACC from uuid: [2011-04-12 15:36:39.861737] I [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: Received ACC from uuid: [2011-04-12 15:36:39.861776] I [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: Received ACC from uuid: [2011-04-12 15:36:39.861809] I [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: Received ACC from uuid: [2011-04-12 15:36:45.725525] I [glusterd-handler.c:1088:glusterd_handle_cli_start_volume] 0-glusterd: Received start vol reqfor volume distribute [2011-04-12 15:36:45.725677] I [glusterd-utils.c:242:glusterd_lock] 0-glusterd: Cluster lock held by 1b6130e7-f83e-4ec6-90f7-46706dbffc4f [2011-04-12 15:36:45.725702] I [glusterd-handler.c:825:glusterd_op_txn_begin] 0-glusterd: Acquired local lock pending frames: patchset: v3.1.3 signal received: 11 time of crash: 2011-04-12 15:36:45 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.1.3 /lib64/libc.so.6(+0x33140)[0x7fc7e2337140] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(+0x2a3e9)[0x7fc7e0b853e9] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_op_sm+0xee)[0x7fc7e0b8b8ce] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_handle_cli_start_volume+0x8a)[0x7fc7e0b8197a] /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x271)[0x7fc7e2cde061] /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_notify+0x93)[0x7fc7e2cde183] /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpc_transport_notify+0x2d)[0x7fc7e2cde77d] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7fc7e09545b4] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_handler+0x19d)[0x7fc7e095476d] /opt/glusterfs/3.1.3/lib64/libglusterfs.so.0(+0x39422)[0x7fc7e2f23422] /opt/glusterfs/3.1.3/sbin/glusterd(main+0x245)[0x405235] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7fc7e2322e5d] /opt/glusterfs/3.1.3/sbin/glusterd[0x402ea9] Restarted all nodes and tried again: patchset: v3.1.3 signal received: 11 time of crash: 2011-04-12 15:47:44 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.1.3 /lib64/libc.so.6(+0x33140)[0x7fb8f65bb140] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(+0x2a3e9)[0x7fb8f4e093e9] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_op_sm+0xee)[0x7fb8f4e0f8ce] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_handle_cli_start_volume+0x8a)[0x7fb8f4e0597a] /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x271)[0x7fb8f6f62061] /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_notify+0x93)[0x7fb8f6f62183] /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpc_transport_notify+0x2d)[0x7fb8f6f6277d] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7fb8f4bd85b4] /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_handler+0x19d)[0x7fb8f4bd876d] /opt/glusterfs/3.1.3/lib64/libglusterfs.so.0(+0x39422)[0x7fb8f71a7422] /opt/glusterfs/3.1.3/sbin/glusterd(main+0x245)[0x405235] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7fb8f65a6e5d] /opt/glusterfs/3.1.3/sbin/glusterd[0x402ea9] I'm going to probably give a 3.1 (qa4) and/or 3.2 (qa11) release a try just in the meantime. Cheers. _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxx http://lists.nongnu.org/mailman/listinfo/gluster-devel
