-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Just wanted to make sure since I didn't catch any mention of these checks. And based on your thoroughness I assume you ran memtest after the ram replacement. What I'd try next in your situation is to boot a different version of the kernel (possibly a different distro) and see if the errors are the same, I'd try something bootable from a DVD or a USB stick. What do you think? On 07/20/2015 11:35 AM, Leslie Rhorer wrote: > On 7/20/2015 3:05 AM, Martin Papik wrote: > > Since you've already found one HW related fault, would you consider > booting into memtest for a couple of passes just to be on the safe > side. > >> I did that after confirming the one stick of memory was bad. >> Twice. I got over 20,000 errors on the bad stick, and 0 on the >> good one. I also swapped the locations on the motherboard, and >> the bad stick still failed while the good one passed 100%. > > And did you by any chance look at SMART if applicable and possibly > running a test on the drives. > >> Yes. SMART found no errors, but think about it. Every time tar >> tries to create a directory when untarring that file in that >> location, the file system croaks when it tries to create a >> directory. Not when reading and not when writing other than when >> it creates a directory. When I create the directory manualy, the >> process quits failing at that point and fails later on during a >> different directory create. The array remains intact when >> reading, and dmesg shows no drive errors. I've re-synced the >> array, which reads every byte on all 8 drives without a single >> mismatch - several times. To my knowledge, no read has ever >> failed except after the filesystem goes offline. I thought >> reads were failing during the CRC checks, but that was a red >> herring. > > Another test I sometimes do when I'm unsure about disks is "cat > /dev/sda > /dev/null" (i.e. a whole disk read test) > >> echo repair > /sys/block/md0/md/sync_action reads not one drive, >> but every byte on all 8 drives. > > and see (dmesg) if any errors show up, unless > >> 'Nary one, and no mismatches. > > you're willing to run badblocks in a read-write nondestructive > mode. In my experience the read test or badblocks can be run > simultaneously with smartctl -t long. But as a start I'd look at > smartctl --all /dev/sd? and see if there are any bad signs. I hope > this helps. Good luck > > > On 07/20/2015 10:41 AM, Leslie Rhorer wrote: >>>> On 7/19/2015 6:27 PM, Dave Chinner wrote: >>>>> On Sat, Jul 18, 2015 at 08:02:50PM -0500, Leslie Rhorer >>>>> wrote: >>>>>> >>>>>> I found the problem with md5sum (and probably nfs, as >>>>>> well). One of the memory modules in the server was bad. >>>>>> The problem with XFS persists. Every time tar tried to >>>>>> create the directory: >>>>> >>>>> Now you need to run xfs_repair. >>>> >>>> I do that every time the array implodes. It makes no >>>> difference. It never mentions cleaning the structure tar >>>> says needs cleaning, and the next time I run tar on that >>>> file, the filesystem craters. >>>> >>>> _______________________________________________ xfs mailing >>>> list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs > >> > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJVrLbVAAoJELsEaSRwbVYr/JoQAKGcNBTtswnSJ9SYpBQMc8aO m2WQaHzLDPkSPLWYeWSGc3clPuf4FdP3A9bDcclCnVV/Ex0WJiCalYfa1Zqpnq5P BinRp1w/cbfTTazLspFT9ySuoloOqNXTPz0MB4uxRTnIDb3Hcahw0O6HhOuZixW3 ocaEOXqVs1cc4YzPwT4Z9aWBEX3ZutMvxNKM4VWT1m8aoRZ3eJMPUKHN04PDUKyT 4Mwilypg9R6r6iberZ9zVwFy0LerElg9Cb90AGLNpyGCutGbOZH7VsoBUTnAmh2E dz4uruFU0x8n87MQccXfSvZQIWG16UDxwjQjEiD4EHtRhYYTNVgq2V8ak94u8w99 0p5WG5+dEnVV0Qgjk2DaZy305LP+5oc2D9GkXJgGTFjMPVV3+9Tnq/XDlm2Hgxn8 hq2q0DoPDQVFMzNLxpGCJfuIdAO3o7z/1rjHpeP2Ol6pPw+hT8SQMehTBU4vMlcp SeZzg485rVtQrWtXVJaRhITAQWSvQxjm9QqLAMdon0oxdKAPZIOtQgr8oEGKgfr7 mknqFPon7sa0c4nAZT7DtTOS+OATbTnYAoUqIuxRf4NCD7dbFUQrccU4/peEE4/H SPzOfgOiAArOVZwWEc7JvydpcKqaEUzYb2KyzsGJFuJHZodrSTzXmUMg/Muc+iQ5 Ao/NeFe/1flevZ060ZEX =1/q4 -----END PGP SIGNATURE----- _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
