Just to give a quick update. I've rebooted the machine and when attempting an xfs_repair I get the following: [root at temporal002 ~]# xfs_repair -n /dev/sdd Phase 1 - find and verify superblock... superblock read failed, offset 0, size 524288, ag 0, rval -1 fatal error -- Input/output error Hardware problem maybe? Unless anyone has a suggestion, I'm pretty much certain at this point that there's no way to recover this data. Even if I could do a dd_rescue and then do a repair on that, gluster would most likely be unhappy with the result. Thanks for everyone's input! Terry -----Original Message----- From: Joe Landman [mailto:landman at scalableinformatics.com] Sent: Monday, March 14, 2011 1:38 PM To: Terry Haley Cc: gluster-users at gluster.org Subject: Re: Quick question regarding xfs_repair On 03/14/2011 01:34 PM, Terry Haley wrote: > Just to clarify. > > I have a shell open that's currently hung doing an umount. So start another > shell and do the lazy umount? > > If that hangs as well, then reboot? Yes. First do an killall -9 umount before the other one. Might not work, but do try it. See if your dmesg output has a call stack at the end indicating a kernel subsystem oops (worse than a file system shutdown). If you have to reboot do this: mount -o remount,sync / which will put the root into synchronous mode (fewer dirty buffers). Then if you have to bounce the unit due to the hang (possible), you can do so with somewhat more safety. > > > Thanks > > -----Original Message----- > From: Joe Landman [mailto:landman at scalableinformatics.com] > Sent: Monday, March 14, 2011 1:28 PM > To: Terry Haley > Cc: gluster-users at gluster.org > Subject: Re: Quick question regarding xfs_repair > > On 03/14/2011 01:22 PM, Terry Haley wrote: > >> At this point, all I can see in my future is trying to reboot without >> remounting and do the repair, which seems like a long shot? >> >> Suggestions? > > Yeah, looks like it couldn't write to the log, so it marked the file > system as down. Believe it or not, this may have saved you ... > > Do a > > umount -l /xfs/mount/point > > and wait a bit. It will do the umount in the background. Put a > "noauto" option on this in the /etc/fstab just in case you need to reboot. > > BTW: Which kernel is this? The stock Centos kernels xfs support comes > from centosplus. Support for xfs isn't bad, but in general, the > RHEL/Centos kernels aren't (in our experience) stable against very heavy > loads, nor are they terribly good with xfs. 5.5 is better. > > Regards > > Joe > > -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
