Hi all, I recently had a machine drop like 14 drives out of a few different member strings in a RAID 60 due to some spurious fault - the drives aren't actually bad. A few of the member strings lost three, four, five drives each and I can't get them or the top level RAID container back up. In the past, when this happened, you could just locate all drives and force the string together with a command like: mdadm --assemble --force /dev/mdN /dev/sdX /dev/sdY ... /dev/sdZ Then, once all component strings were forced back together, force back the top level container and you'd be good to go. It seems like in recent kernel or md versions, now, when this happens, the spuriously ejected drives end up stuck in State "active" instead of State "clean" which prevents this method of recovery from working. I am making an assumption that the State is maybe just a single bit or a few bits worth of flags in the RAID superblock on each physical disk. What's the offset from beginning of disk for the State flag such that I can maybe go in with a hex editor on the physical drive and toggle it and try to force the array back together? I'm looking in mdadm and I don't see any mechanism in there to forcibly toggle the drive state. Maybe this is a feature that could be added in the future for desperate recovery purposes like this? Is toggling the State on individual member drives not as simple as I'm thinking? Of course, no backups of this LUN, so trying to do anything possible to jam the strings back together and get this thing back online. Thank you! Sean
