since the VM files (VHD) are available on LVM - can a new gluster volume be created and exported over NFS - without risk of dataloss? ________________________________ From: srinivas jonn <jmsrinivas at yahoo.com> To: Krishnan Parthasarathi <kparthas at redhat.com>; "gluster-users at gluster.org" <gluster-users at gluster.org> Sent: Monday, 3 June 2013 11:42 PM Subject: Re: gluster startup failure Hello Gluster users, thought of posing a more refined question, thanks to the support of Krish. problem statement: Gluster volume start - failure RPM installation of 3.3.0 on CentOS 6.1 - XFS is filesystem layer - NFS export , distributed single node, TCP ? this server has experienced a accidental powerloss while in operation. any help in resolving or debug issue is appreciated. ? glusterd logs indicate failure to resolve the brick: ? [2013-06-03 12:03:24.660330] I [glusterd-volume-ops.c:290:glusterd_handle_cli_start_volume] 0-glusterd: Received start vol reqfor volume gvol1 [2013-06-03 12:03:24.660384] I [glusterd-utils.c:285:glusterd_lock] 0-glusterd: Cluster lock held by 16ee7a4e-ee9b-4543-bd61-9b444100693d [2013-06-03 12:03:24.660398] I [glusterd-handler.c:463:glusterd_op_txn_begin] 0-management: Acquired local lock [2013-06-03 12:03:24.842904] E [glusterd-volume-ops.c:842:glusterd_op_stage_start_volume] 0-: Unable to resolve brick 10.0.0.30:/export/brick1 [2013-06-03 12:03:24.842938] E [glusterd-op-sm.c:1999:glusterd_op_ac_send_stage_op] 0-: Staging failed [2013-06-03 12:03:24.842959] I [glusterd-op-sm.c:2039:glusterd_op_ac_send_stage_op] 0-glusterd: Sent op req to 0 peers [2013-06-03 12:03:24.842982] I [glusterd-op-sm.c:2653:glusterd_op_txn_complete] 0-glusterd: Cleared local lock From: Krishnan Parthasarathi <kparthas at redhat.com> To: srinivas jonn <jmsrinivas at yahoo.com> Sent: Monday, 3 June 2013 4:24 PM Subject: Re: recovering gluster volume || startup failure Is this a source install or an rpm install? If it is a source install, the logs would be present under <install-prefix>/var/log/glusterfs Having said that, could you attach etc-glusterfs-glusterd.log file? Does the gluster CLI print any error messages to the terminal, when volume-start fails? thanks, krish ----- Original Message ----- > there is no /var/log/glusterfs/.cmd_log_history file . > > gluster volume start <volume> - volume start has been unsuccessful > > > let me know for any specific log, I am trying to debug why volume is not > starting - > > feel free to copy the gluster-user DL if you think right > > > > ________________________________ >? From: Krishnan Parthasarathi <kparthas at redhat.com> > To: srinivas jonn <jmsrinivas at yahoo.com> > Cc: gluster-users at gluster.org > Sent: Monday, 3 June 2013 3:56 PM > Subject: Re: recovering gluster volume || startup failure >? > > Did you run "gluster volume start gvol1"? Could you attach > /var/log/glusterfs/.cmd_log_history (log file)? > From the logs you have pasted, it looks like volume-stop is the last command > you executed. > > thanks, > krish > > ----- Original Message ----- > > the volume is not starting - this was the issue.. please let mw know the > > diagnostic or debug procedures, > > ? > > ? > > logs: > > ? > > usr/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x293) [0x30cac0a443] > > ?/usr/sbin/glusterfsd(glusterfs_handle_terminate+0x15) [0x40a955]))) 0-: > > ?received signum (15), shutting down > > ?[2013-06-02 09:32:16.973895] W [glusterfsd.c:831:cleanup_and_exit] > > ?(-->/lib64/libc.so.6(clone+0x6d) [0x3ef56e68ed] > > ?(-->/lib64/libpthread.so.0() > > ?[0x3ef5a077e1] (-->/usr/sbin/glusterfsd(glusterfs_sigwaiter+0xdd) > > ?[0x405d4d]))) 0-: received signum (15), shutting down > >? > > > > ________________________________ > >? From: Krishnan Parthasarathi <kparthas at redhat.com> > > To: srinivas jonn <jmsrinivas at yahoo.com> > > Cc: gluster-users at gluster.org > > Sent: Monday, 3 June 2013 3:27 PM > > Subject: Re: recovering gluster volume || startup failure > >? > > > > Srinivas, > > > > The volume is in stopped state. You could start the volume by running > > "gluster volume start gvol1". This should make your attempts at mounting > > the volume successful. > > > > thanks, > > krish > > > > ----- Original Message ----- > > > Krish, > > > this is giving general volume information , can the state of volume known > > > from any specific logs? > > > #gluster volume info gvol1 > > > Volume Name: gvol1 > > > Type: Distribute > > > Volume ID: aa25aa58-d191-432a-a84b-325051347af6 > > > Status: Stopped > > > Number of Bricks: 1 > > > Transport-type: tcp > > > Bricks: > > > Brick1: 10.0.0.30:/export/brick1 > > > Options Reconfigured: > > > nfs.addr-namelookup: off > > > nfs.port: 2049 > > > From: Krishnan Parthasarathi <kparthas at redhat.com> > > > To: srinivas jonn <jmsrinivas at yahoo.com> > > > Cc: gluster-users at gluster.org > > > Sent: Monday, 3 June 2013 3:14 PM > > > Subject: Re: recovering gluster volume || startup failure > > > > > > Srinivas, > > > > > > Could you paste the output of "gluster volume info gvol1"? > > > This should give us an idea as to what was the state of the volume > > > before the power loss. > > > > > > thanks, > > > krish > > > > > > ----- Original Message ----- > > > > Hello Gluster users: > > > > sorry for long post, I have run out of ideas here, kindly let me know > > > > if > > > > i > > > > am > > > > looking at right places for logs and any suggested actions.....thanks > > > > a sudden power loss casued hard reboot - now the volume does not start > > > > Glusterfs- 3.3.1 on Centos 6.1 transport: TCP > > > > sharing volume over NFS for VM storage - VHD Files > > > > Type: distributed - only 1 node (brick) > > > > XFS (LVM) > > > > mount /dev/datastore1/mylv1 /export/brick1 - mounts VHD files.......is > > > > there > > > > a way to recover these files? > > > > cat export-brick1.log > > > > [2013-06-02 09:29:00.832914] I [glusterfsd.c:1666:main] > > > > 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version > > > > 3.3.1 > > > > [2013-06-02 09:29:00.845515] I [graph.c:241:gf_add_cmdline_options] > > > > 0-gvol1-server: adding option 'listen-port' for volume 'gvol1-server' > > > > with > > > > value '24009' > > > > [2013-06-02 09:29:00.845558] I [graph.c:241:gf_add_cmdline_options] > > > > 0-gvol1-posix: adding option 'glusterd-uuid' for volume 'gvol1-posix' > > > > with > > > > value '16ee7a4e-ee9b-4543-bd61-9b444100693d' > > > > [2013-06-02 09:29:00.846654] W [options.c:782:xl_opt_validate] > > > > 0-gvol1-server: option 'listen-port' is deprecated, preferred is > > > > 'transport.socket.listen-port', continuing with correction > > > > Given volfile: > > > > +------------------------------------------------------------------------------+ > > > > 1: volume gvol1-posix > > > > 2: type storage/posix > > > > 3: option directory /export/brick1 > > > > 4: option volume-id aa25aa58-d191-432a-a84b-325051347af6 > > > > 5: end-volume > > > > 6: > > > > 7: volume gvol1-access-control > > > > 8: type features/access-control > > > > 9: subvolumes gvol1-posix > > > > 10: end-volume > > > > 11: > > > > 12: volume gvol1-locks > > > > 13: type features/locks > > > > 14: subvolumes gvol1-access-control > > > > ---------- > > > > ----------------- > > > > > > > > 46: option transport-type tcp > > > > 47: option auth.login./export/brick1.allow > > > > 6c4653bb-b708-46e8-b3f9-177b4cdbbf28 > > > > 48: option auth.login.6c4653bb-b708-46e8-b3f9-177b4cdbbf28.password > > > > 091ae3b1-40c2-4d48-8870-6ad7884457ac > > > > 49: option auth.addr./export/brick1.allow * > > > > 50: subvolumes /export/brick1 > > > > 51: end-volume > > > > +------------------------------------------------------------------------------+ > > > > [2013-06-02 09:29:03.963001] W [socket.c:410:__socket_keepalive] > > > > 0-socket: > > > > failed to set keep idle on socket 8 > > > > [2013-06-02 09:29:03.963046] W > > > > [socket.c:1876:socket_server_event_handler] > > > > 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported > > > > [2013-06-02 09:29:04.850120] I > > > > [server-handshake.c:571:server_setvolume] > > > > 0-gvol1-server: accepted client from > > > > iiclab-oel1-9347-2013/06/02-09:29:00:835397-gvol1-client-0-0 (version: > > > > 3.3.1) > > > > [2013-06-02 09:32:16.973786] W [glusterfsd.c:831:cleanup_and_exit] > > > > (-->/usr/lib64/libgfrpc.so.0(rpcsvc_notify+0x93) [0x30cac0a5b3] > > > > (-->/usr/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x293) > > > > [0x30cac0a443] > > > > (-->/usr/sbin/glusterfsd(glusterfs_handle_terminate+0x15) [0x40a955]))) > > > > 0-: > > > > received signum (15), shutting down > > > > [2013-06-02 09:32:16.973895] W [glusterfsd.c:831:cleanup_and_exit] > > > > (-->/lib64/libc.so.6(clone+0x6d) [0x3ef56e68ed] > > > > (-->/lib64/libpthread.so.0() > > > > [0x3ef5a077e1] (-->/usr/sbin/glusterfsd(glusterfs_sigwaiter+0xdd) > > > > [0x405d4d]))) 0-: received signum (15), shutting down > > > > NFS LOG > > > > [2013-06-02 09:29:00.918906] I [rpc-clnt.c:1657:rpc_clnt_reconfig] > > > > 0-gvol1-client-0: changing port to 24009 (from 0) > > > > [2013-06-02 09:29:03.963023] W [socket.c:410:__socket_keepalive] > > > > 0-socket: > > > > failed to set keep idle on socket 8 > > > > [2013-06-02 09:29:03.963062] W > > > > [socket.c:1876:socket_server_event_handler] > > > > 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported > > > > [2013-06-02 09:29:04.849941] I > > > > [client-handshake.c:1636:select_server_supported_programs] > > > > 0-gvol1-client-0: > > > > Using Program GlusterFS 3.3.1, Num (1298437), Version (330) > > > > [2013-06-02 09:29:04.853016] I > > > > [client-handshake.c:1433:client_setvolume_cbk] > > > > 0-gvol1-client-0: Connected to 10.0.0.30:24009, attached to remote > > > > volume > > > > '/export/brick1'. > > > > [2013-06-02 09:29:04.853048] I > > > > [client-handshake.c:1445:client_setvolume_cbk] > > > > 0-gvol1-client-0: Server and Client lk-version numbers are not same, > > > > reopening the fds > > > > [2013-06-02 09:29:04.853262] I > > > > [client-handshake.c:453:client_set_lk_version_cbk] 0-gvol1-client-0: > > > > Server > > > > lk version = 1 > > > > > > > > _______________________________________________ > > > > Gluster-users mailing list > > > > Gluster-users at gluster.org > > > > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > > > > > > > > > > > _______________________________________________ > > > Gluster-users mailing list > > > Gluster-users at gluster.org > > > http://supercolony.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130604/5a326887/attachment-0001.html>
