On 15/06/16 10:49, Nicolas Noble wrote:
I
think in your case you're better off stopping an array that has less than
parity drives than it should, either using a udev rule or using mdadm
--monitor.
I actually have been unsuccessful in these attempts so far. What
happens is that you very quickly get processes that get indefinitely
stuck (indefinitely as in 'waiting on a very very long kernel
timeout') trying to write something, so that the ext4fs layer becomes
unresponsive on these threads, or take a very long time. Killing the
processes takes a very long time because they are stuck in a kernel
operation. And if potentially more processes can spawn back up, the
automated script starts an interesting game of whack-a-mole in order
to unmount the filesystem.
And you can't stop the underlying arrays without first stopping the
whole chain (umount, stop the lvm volume, etc...), otherwise you
simply get "device is busy" errors, hence the whack-a-mole process
killing. The only working method I've managed to successfully
implement is to programatically loop over the list of all the drives
involved in the filesystem, on all the raids involved, and flag all of
them as failed drives. This way, you get to really put "emergency
brakes" on. I find that to be a very, very scary method however.
I understand your concern, but I remember a thread where it was
explained that a RAID0 or linear one basically behaves like a hard drive
would: since there is no parity and the data is distributed, if say half
of the devices of the RAID0 are unavailable the LBAs on the other half
of that RAID will work fine, like if you had a SSD with half of its
cells broken. So your issue seems to be more related with dealing with
IO errors than anything. I would imagine that if the filesystem's
superblock was to become unread/writeable (if it was on the missing
RAID) then the filesystem would "fail" (be remounted as readonly). Other
than that there's not much to be done apart from instructing your
program to stop the IOs and/or fiddling with timeouts to speed up the
failure of the process.
The "emergency brake" as you put it would work similarly to a RAID5
losing more than it can: the array will error every write sent to it.
Alternatively you could disconnect the drives from Linux using the
"delete" sysfs property. If you use a journalled filesystem you
shouldn't lose any data over any of this anyways, so that seems safe.
HTH,
Regards,
Ben.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
