[Please trim replies/avoid top-posting] On 09/22/2013 06:45 PM, Dave Gomboc wrote: > Yikes. The data had been accessible on the evening of Sat. Sept 14. > I had (apparently falsely) hoped that there were two complete copies > of all the data, and that to restore access it would be enough to > assign the correct drives to position 1 and 2 based on what matched > against the contents of positions 0 and 3. We will certainly want to inspect your drives to try to determine which ones belong where. > I have ran these commands on the four 4TB drives that are the copies. > I think the smartctl -x commands in particular you want to see run > against the original set of drives, so after sending this message, I > will take down the machine, swap out the four copied drives for the > four original drives, then run these commands again on the original > drives. Not really. Once we start poking at the devices at a low level, the most important thing is to keep track of drive serial numbers versus linux device name. I created a utility to help document a running system [1], but that wouldn't be much help here. > I'm not knowledgeable about the LVM backup file. Would one typically > happen to be stored in /boot? That was the only directory I had that > was not part of the raid10 system. I had distinct logical volumes for > home, opt, usr, var, srv, root (/), and tmp. No, on my systems they are stored under /etc/lvm/backup/. You don't have your root volume, so you can only use the backup embedded in the PV metadata. > SCT Error Recovery Control: > Read: Disabled > Write: Disabled While you are fixing things, you *really* need to fix this. Consumer drives don't wake up with this enabled, if supported at all. You need to set these in a boot script. Search the archives for various combinations of "scterc", "error recovery control", "tler", and "ure" for a detailed explanation. Now, please show the hex dump of your superblocks and the beginning of the data area on each partition: dd if=/dev/sdX1 bs=4096 skip=1 count=1 2>/dev/null |hexdump -C dd if=/dev/sdX1 bs=4096 skip=2048 count=16 2>/dev/null |hexdump -C And reverify which drive name in linux corresponds to which serial number. Phil [1] http://github.com/pturmel/lsdrv -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html