I found that the promise array had been restarted via watchdog timer. I am investigating that avenue via promise (albeit slow). Note: the watchdog reset the controller days after the initial ext4 messages. I'm not saying they are unrelated. I just what to get all of the facts out there. I suspect the connection between the server and the promise got hosed when the controller was reset. When I restart the server, I could fsck the drive. The fsck is currently running (and has been for some time now). It is doing a ton of "Inode ######## ref count is 2, should be 1. Fix? yes" "Unattached inode #########" "Connect to /lost+found? yes" I am running fsck in a script session; however, there are currently a ton of the messages above (current log size: 106M). Do you think it is still hardware? If so, is there a command that would stress it enough to break quickly? What is the best way to isolate hardware problems? Bryan From: Eric Sandeen <sandeen@xxxxxxxxxx> To: bryan.coleman@xxxxxxxx Cc: linux-ext4@xxxxxxxxxxxxxxx, "Ted Ts'o" <tytso@xxxxxxx> Date: 02/08/2011 10:21 AM Subject: Re: ext4 problems with external RAID array via SAS connection Sent by: linux-ext4-owner@xxxxxxxxxxxxxxx On 2/8/11 8:50 AM, bryan.coleman@xxxxxxxx wrote: > Well, I attempted to run fsck on the problem drive using the script > command to capture the transcript; however, it failed to read a block from > the file system. The exception was "fsck.ext4: Attempt to read block from > filesystem resulted in short read while trying to open > /dev/mapper/vg_storage-lv_storage". > > Other messages that are now in /var/log/messages: > > Buffer I/O error on device dm-2, logical block 0 > lost page write due to I/O error on dm-2 > EXT4-fs (dm-2): previous I/O error to superblock detected > Buffer I/O error on device dm-2, logical block 0 > lost page write due to I/O error on dm-2 > Buffer I/O error on device dm-2, logical block 0 > Buffer I/O error on device dm-2, logical block 1 > Buffer I/O error on device dm-2, logical block 2 > Buffer I/O error on device dm-2, logical block 3 > Buffer I/O error on device dm-2, logical block 0 > EXT4-fs (dm-2): unable to read superblock > > > Since it looks like I need to start the process all over again, is there a > good way to quickly determine if the problem is hardware related? Is > there a preferred method that will stress test the drive and shed more > light on what might be going wrong? You have a hardware problem... "Buffer I/O error on device dm-2, logical block 0" means that you failed to read the first block on that device; not something e2fsck can fix, I'm afraid; you'll need to sort out what's wrong with the storage, first. -Eric > Thank you, > > Bryan > > > > From: bryan.coleman@xxxxxxxx > To: linux-ext4@xxxxxxxxxxxxxxx, linux-ext4-owner@xxxxxxxxxxxxxxx > Date: 02/08/2011 08:19 AM > Subject: Re: ext4 problems with external RAID array via SAS > connection > Sent by: linux-ext4-owner@xxxxxxxxxxxxxxx > > > > When I ran fsck after the first bout of failure, it did report a lot of > errors. I do not have a copy of that fsck transcript; however, I have not > > yet run fsck since my second attempt. Is there a method of capturing the > transcript that is preferred? > > Bryan > > > > From: Ted Ts'o <tytso@xxxxxxx> > To: bryan.coleman@xxxxxxxx > Cc: linux-ext4@xxxxxxxxxxxxxxx > Date: 02/07/2011 05:55 PM > Subject: Re: ext4 problems with external RAID array via SAS > connection > Sent by: linux-ext4-owner@xxxxxxxxxxxxxxx > > > > On Mon, Feb 07, 2011 at 01:53:18PM -0500, bryan.coleman@xxxxxxxx wrote: >> I am experiencing problems with an ext4 file system. >> >> At first, the drive seemed to work fine. I was primarily copying things > > >> to the drive migrating data from another server. After many GBs of > data, >> that seemingly successfully were done being transferred, I started > seeing >> ext4 errors in /var/log/messages. I then unmounted the drive and ran > fsck >> on it (which took multiple hours to run). I then ls'ed around and one > of >> the areas caused the system to again throw ext4 errors. > > Did fsck report any errors? Do you have a copy of your fsck > transcript? > > The errors you've reported do make me suspicious that there's > something unstable with your hardware... > > - Ted > -- -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
