Hi - > I'm going to probably give a 3.1 (qa4) and/or 3.2 (qa11) release a try Please try with stable version(3.1.x) instead of (3.1.xQAx) QA releases. Also provide the output of command - "gluster volume info" along with server and client logs. -- ---- Cheers, Lakshmipathi.G FOSS Programmer. ----- Original Message ----- From: "Mark \"Naoki\" Rogers" <mrogers@xxxxxxxxxxxxxxxxxxx> To: gluster-devel@xxxxxxxxxx Sent: Tuesday, April 12, 2011 12:48:34 PM Subject: Re: 3.1.3 crash on volume start Sadly no go with 3.1.4qa3: patchset: v3.1.4qa3 signal received: 11 time of crash: 2011-04-12 16:16:34 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.1.4qa3 /lib64/libc.so.6(+0x33140)[0x7f68d48c8140] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/xlator/mgmt/glusterd.so(+0x3323b)[0x7f68d311e23b] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/xlator/mgmt/glusterd.so(glusterd_op_sm+0xee)[0x7f68d311d6ce] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/xlator/mgmt/glusterd.so(glusterd_handle_cli_start_volume+0x8f)[0x7f68d310c91f] /opt/glusterfs/3.1.4qa3/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x28f)[0x7f68d526e01f] /opt/glusterfs/3.1.4qa3/lib64/libgfrpc.so.0(rpcsvc_notify+0x123)[0x7f68d526e653] /opt/glusterfs/3.1.4qa3/lib64/libgfrpc.so.0(rpc_transport_notify+0x2d)[0x7f68d527221d] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7f68d2ee3b44] /opt/glusterfs/3.1.4qa3/lib64/glusterfs/3.1.4qa3/rpc-transport/socket.so(socket_event_handler+0xb3)[0x7f68d2ee3de3] /opt/glusterfs/3.1.4qa3/lib64/libglusterfs.so.0(+0x37d42)[0x7f68d54b2d42] /opt/glusterfs/3.1.4qa3/sbin/glusterd(main+0x263)[0x4054b3] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7f68d48b3e5d] /opt/glusterfs/3.1.4qa3/sbin/glusterd[0x402db9] On 04/12/2011 04:07 PM, Mark "Naoki" Rogers wrote: > I had an interesting situation with a (3x3) distribute+replicate setup. > One of the machines died (memory problem, hard lock) and gluster didn't > exactly like it (although that's another story). > > Anyway I decided to keep on truckin' and use a 5 node distribute (since > this is mainly a network test anyway) and the volume creation was ok but > startup failed with the old "cli op: 16" message (note: would be nice if > that actually said lock file issue if that's what it is - seems to be > solved by a restart anyway). > > I restarted glusterd but everytime I try and "volume start" it just > crashes immediately: > > [2011-04-12 15:36:39.861099] I > [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: > af4007ac-804c-433f-a6e9-0a7d6cafe432, host: pdbsearch12 > [2011-04-12 15:36:39.861123] I > [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: > 5e4ef932-cca7-4440-a8e3-4415b84c2307, host: pdbsearch13 > [2011-04-12 15:36:39.861148] I > [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: > df6c1fb1-cf5d-4a6b-ba15-fb26319e37db, host: pdbsearch14 > [2011-04-12 15:36:39.861172] I > [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: > 2cc51957-2c92-4d24-a53c-c7ded19e80b7, host: pdbsearch15 > [2011-04-12 15:36:39.861695] I > [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: > Received ACC from uuid: > [2011-04-12 15:36:39.861737] I > [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: > Received ACC from uuid: > [2011-04-12 15:36:39.861776] I > [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: > Received ACC from uuid: > [2011-04-12 15:36:39.861809] I > [glusterd-rpc-ops.c:626:glusterd3_1_friend_update_cbk] 0-glusterd: > Received ACC from uuid: > [2011-04-12 15:36:45.725525] I > [glusterd-handler.c:1088:glusterd_handle_cli_start_volume] 0-glusterd: > Received start vol reqfor volume distribute > [2011-04-12 15:36:45.725677] I [glusterd-utils.c:242:glusterd_lock] > 0-glusterd: Cluster lock held by 1b6130e7-f83e-4ec6-90f7-46706dbffc4f > [2011-04-12 15:36:45.725702] I > [glusterd-handler.c:825:glusterd_op_txn_begin] 0-glusterd: Acquired > local lock > pending frames: > > patchset: v3.1.3 > signal received: 11 > time of crash: 2011-04-12 15:36:45 > configuration details: > argp 1 > backtrace 1 > dlfcn 1 > fdatasync 1 > libpthread 1 > llistxattr 1 > setfsid 1 > spinlock 1 > epoll.h 1 > xattr.h 1 > st_atim.tv_nsec 1 > package-string: glusterfs 3.1.3 > /lib64/libc.so.6(+0x33140)[0x7fc7e2337140] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(+0x2a3e9)[0x7fc7e0b853e9] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_op_sm+0xee)[0x7fc7e0b8b8ce] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_handle_cli_start_volume+0x8a)[0x7fc7e0b8197a] > /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x271)[0x7fc7e2cde061] > /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_notify+0x93)[0x7fc7e2cde183] > /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpc_transport_notify+0x2d)[0x7fc7e2cde77d] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7fc7e09545b4] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_handler+0x19d)[0x7fc7e095476d] > /opt/glusterfs/3.1.3/lib64/libglusterfs.so.0(+0x39422)[0x7fc7e2f23422] > /opt/glusterfs/3.1.3/sbin/glusterd(main+0x245)[0x405235] > /lib64/libc.so.6(__libc_start_main+0xfd)[0x7fc7e2322e5d] > /opt/glusterfs/3.1.3/sbin/glusterd[0x402ea9] > > Restarted all nodes and tried again: > > patchset: v3.1.3 > signal received: 11 > time of crash: 2011-04-12 15:47:44 > configuration details: > argp 1 > backtrace 1 > dlfcn 1 > fdatasync 1 > libpthread 1 > llistxattr 1 > setfsid 1 > spinlock 1 > epoll.h 1 > xattr.h 1 > st_atim.tv_nsec 1 > package-string: glusterfs 3.1.3 > /lib64/libc.so.6(+0x33140)[0x7fb8f65bb140] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(+0x2a3e9)[0x7fb8f4e093e9] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_op_sm+0xee)[0x7fb8f4e0f8ce] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/xlator/mgmt/glusterd.so(glusterd_handle_cli_start_volume+0x8a)[0x7fb8f4e0597a] > /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x271)[0x7fb8f6f62061] > /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpcsvc_notify+0x93)[0x7fb8f6f62183] > /opt/glusterfs/3.1.3/lib64/libgfrpc.so.0(rpc_transport_notify+0x2d)[0x7fb8f6f6277d] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7fb8f4bd85b4] > /opt/glusterfs/3.1.3/lib64/glusterfs/3.1.3/rpc-transport/socket.so(socket_event_handler+0x19d)[0x7fb8f4bd876d] > /opt/glusterfs/3.1.3/lib64/libglusterfs.so.0(+0x39422)[0x7fb8f71a7422] > /opt/glusterfs/3.1.3/sbin/glusterd(main+0x245)[0x405235] > /lib64/libc.so.6(__libc_start_main+0xfd)[0x7fb8f65a6e5d] > /opt/glusterfs/3.1.3/sbin/glusterd[0x402ea9] > > > I'm going to probably give a 3.1 (qa4) and/or 3.2 (qa11) release a try > just in the meantime. > Cheers. > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxx > http://lists.nongnu.org/mailman/listinfo/gluster-devel _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxx http://lists.nongnu.org/mailman/listinfo/gluster-devel
