On Thu, Sep 25, 2014 at 07:30:23AM +0000, Simon Dray wrote: > Dear Sirs > > I wonder if you can help with an issue we see re-occuring on a regular basis with one of our HP systems which uses a HP 420 Raid controller tl;dr: more information is needed about your system to make sense of the problem. See here: http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F > Action taken > > We first saw the following: > [root@ content]# ls > ls: cannot open directory .: Input/output error The filesystem has shut down due to a fatal error. > We try to run: > [root@ /]# xfs_check /dev/md0 > xfs_check: /dev/md0 contains a mounted and writable filesystem > fatal error -- couldn't initialize XFS library FYI, xfs_check was deprecated a quite a while ago. It no longer exists in current releases.... > We also tried to umount the /dev/md0 before runniing xfs_check but no luck. We received the error: device is in use That can happen if the storage has gone bad and IOs have been lost. > We use xfs for one of our large raid file systems and we are seeing the xfs filesystem go offline with the following messages in dmesg > > messages-20140921:Sep 18 23:01: kernel: XFS (md0): Device md0: metadata write error block 0x5e28623d8 What messages occurred before this? Something reported an IO error back to XFS, and so that something should have logged an error message... > messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): I/O error occurred: meta-data dev md0 block 0x445cccc40 ("xlog_iodone") error 5 buf count 32768 > messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): xfs_do_force_shutdown(0x2) called from line 891 of file fs/xfs/xfs_log.c. Return address = 0xffffffffa2c428dc > messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): Log I/O Error Detected. Shutting down filesystem > messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): Please umount the filesystem and rectify the problem(s) > messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): xfs_imap_to_bp: xfs_trans_read_buf() returned error 5. > messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): xfs_iunlink_remove: xfs_itobp() returned error 5. Yup, kernel code is at least 2 years old, because we removed xfs_itobp in mid 2012... ;) > In all occurrences the only way to recover from this is to reboot > the system and allow xfs_repair to run during boot this clears the > issue until next time > > We have checked the RAID health and nothing seems to be amiss, if > you could help with this it would be much appreciated That's par for the course when hardware raid goes AWOL - they almost never report that they had a problem when they hang (e.g. firmware crashes so can't log an event to say it crashed). But really, more information about your system and more complete logs are needed to be able to make any progress triaging the problem. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
