On 18/01/2022 23:00, NeilBrown wrote:
Firstly, given that superblocks seem to disappear every now and then, does anybody have any ideas for something that might help us track it down? The 1.2 superblock is 4K into the device I believe? So if I copy the first 8K ( dd if=/dev/sda4 of=sda4.img bs=4K count=2 ) of each partition, that might help provide any clues as to what's happened to it? What am I looking for? What is the superblock supposed to look like?
Yes, 4K offset. Yes, that dd command will get what you want it to. It hardly matters what the superblock should looks like, because it won't be there. The thing you want to know is: what is there? i.e. you see random bytes and need to guess what they mean, so you can guess where they came from. Best to post the "od -x" output and crowd-source.
That's exactly what I was thinking. But I was thinking if it had been damaged rather than destroyed maybe stuff would have been recoverable.
Are you sure the partition starts haven't changed? Was the array made of whole-devices or of partitions?
That's what I missed. I forgot my array was on top of dm-integrity, so although I think of it as sda4, sdb1, sdc4, they each in fact have an extra layer between them and the raid.
Dunno what or why, but my systemd service that fires that up failed. status tells me it was killed after 2msec.
So if that wasn't running, the integrity devices weren't there, and mdadm couldn't start the array.
Oh well, the good thing is that backup drive is on its way. I'm planning to put plain lvm on it, and write a bunch of services that create backup volumes then do a overwrite-in-place rsync. So as I keep advising people, it does an incremental backup, but the COW volumes mean I have full backups.
Cheers, Wol