On Tue, Nov 25, 2014 at 16:27:07 CET, Sven Eschenberg wrote: > I think Mark was aiming at some other concerns with his question. > > As you stated, backups are mandatory and RAID's purpose is extended > availability (and speed). > > Regarding the concerns of the OP: > When a device fails and gets marked as failed there's no difference to > single drive operation. With TLER drives the drive will probably not get > marked faulty and the broken sector can be rewritten with the data of the > other leg, if that's implemented apropriately. > What is problematic in a RAID is failure and unreported errors during > read(). Say a sector including the LUKS header is instable, gets read and > the retrieved data is faulty then broken data might get written to the > mirror during manipulation operations including a following write. (Can be > compensated by backups though) That should noty happen unless there is a severe error in the driver, or unless the disk itself did not notice. For spinning disk, something like this will only happen if the memory or controller of the disk is going bad in a sublte way (very, very rare). SSDs are still much worse in that regard, due to less firmware maturity. Still, even here it is not much of a concern. AFAIK, md-raid does not refresh sectors from other disks, it simply reads them from there and you should get error reports both from SMART monitoring and from the disk driver (in thesystem log), as you have an "uncorrectable sector" and you should get an error from RAID monitoring as well. So, unless I am mistaken about md-raid, this is NOT a concern, unless you run disks and RAID without monitoring, in which case you will always be surprised by disk failures even with RAID. As the probablility of the LUKS header being hit is small (with a 2TB disk, e.g., the probability of a random bad sector being in the LUKS header is 0.0001%, small enough to be irrelevant), this is not even a real concern without monitoring unless you really are oblivous about your disk dying or corrupting data. This is a disk reliability problem, do not solve it on LUKS layer, but on raw-disk and RAID layer. > With two disks the probability of such a specific error increases, on the > other hand a RAID1 implementation *should* level reads which in turn > decreases the prob. to hit such a specific read error. No. Not unless the RAID layer is too smart for its own good. AFAIK, RAID implementations that do refresh data have extra checksums to be really, really sure they do not silently corrypt data. For a while (long ago), you could buy enterprise disks with 540 Byte sectors for that purpose. > The question that remains is: How probable is an unnoted (or unreported) > read error and how does the RAID implementation handle specific error > scenarios? (Unfortunately there's firmware bugs ...) > > Say the mirrors are incosistent due to an unnoted read error, the RAID > layer can not decide which of the two legs has faulty data. It basically always can, as the disks know. Unrecognized disk read errors are so unlikely as to be irrelevant. Of course bad RAM or CPU (on the disk or in your computer) can cause all sorts of havoc, but again, that is not something you can solve on the LUKS layer. > It can whatsoever reread both legs in hope the faulty read is corrected on > reread and rewrite afterwards. I fear such actions are only taken during > a forced rebuild though. Unless you have TLER disks, it is always taken, as the disk itself does it. Really, if you are concerned about disks dying (and you should be), do the sane thing and have SMART monitoring and regular long SMART selftests (I do them every 14 days) via smartmond. If you are concerned about the RAID being inconsistent (and you should be), do RAID consistency checks (I do them every 7 days) and RAID monitoring via mdadm. Do not mess with things on the crypto-layer that do not belong there. That can only make things worse. Gr"usse, Arno > Reagrds > > -Sven > > On Tue, November 25, 2014 15:24, Arno Wagner wrote: > > On Tue, Nov 25, 2014 at 11:28:47 CET, Fabrice Bongartz wrote: > >> Hi Mark, > >> > >> I currently employ the following setup: > >> I have multiple md software raid 1 arrays and luks on top of that. For > >> example, /dev/sda1 and /dev/sdb1 are two identifcal disks which are in a > >> raid1 using md raid as /dev/md0. The luks encrypted device is /dev/md0. > >> So far, I have had two discs fail in two different arrays and I have had > >> no problem restoring them. The array continued in degrated mode and I > >> could safely replace the two drives and add the new disks to the arrays > >> using the mdadm command. > >> > >> I am also curious as to what the devs have to say about this. > > > > RAID and LUKS are in separate layers and do not influence > > each other. See FAQ Items 2.2 ad 2.8. 2.8 also has a picture. > > > > If you place LUKS atop RAID, you get pretty much > > the same change as with a normal filesystem atop RAID. Of > > course, the LUKS header is critical, which is why you should > > always have a header backup, just the same as without RAID. > > > > If you place LUKS below RAID (not that good an idea), you > > will have to unlock the raw devices before the RAID can > > be assembled. You should have header backups for as much > > devices as are neded to assemble the RAID, but better for > > all. > > > > Really, these are separate issuses, LUKS and RAID do not > > magically interact behind your back. > > > > Gr"usse, > > Arno > > > >> BTW: I always make a complete backup on a third external disk, I don't > >> want to take any chances. > >> > >> Cheers, > >> > >> Fabrice Bongartz > >> > >> > >> Von: "Mark Connor" <markc44@xxxxxxx> > >> An: "dm-crypt" <dm-crypt@xxxxxxxx> > >> Gesendet: Dienstag, 25. November 2014 11:03:17 > >> Betreff: LUKS safety on RAID 1 mirror > >> > >> Hello > >> > >> I currently have a deployment with luks (aes-cbc-256) on different 1TB, > >> 500GB, 300GB etc. drives. All the drives use different keys and XFS > >> filesystem on the top of luks. > >> I'm planning to replace this setup with 2X4TB disks in software raid1 > >> (with mdraid) but I have my concerns. > >> > >> 1, If a sector goes bad on disk1 that normally shouldn't be replicated > >> to disk2 but in case of luks I don't know what happens then. > >> > >> 2, I think it is more practical -when one is dealing with encryption- to > >> keep many smaller partitions encrypted with separate keys, in case of > >> partial disk failure (other parts of the disk can still be accessed). > >> Also all the partitions have their own separate luks headers... > >> > >> Unlike if I don't even create partition just put sda (4TB) sdb(4TB) into > >> and md0 array and make luks on that one, if anything goes wrong with the > >> header I lose all my data or if any part of the disks breaks. > >> > >> I know that ultimately raid is only protect against drive failures (not > >> if files get corrupted or deleted) so have to have a separated > >> snapshotted backup next to it. But would implementing raid1 in case of > >> luks be an advantage or a disadvantage? > >> > >> Thanks > >> _______________________________________________ > >> dm-crypt mailing list > >> dm-crypt@xxxxxxxx > >> http://www.saout.de/mailman/listinfo/dm-crypt > > > >> _______________________________________________ > >> dm-crypt mailing list > >> dm-crypt@xxxxxxxx > >> http://www.saout.de/mailman/listinfo/dm-crypt > > > > > > -- > > Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@xxxxxxxxxxx > > GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D > > 9718 > > ---- > > A good decision is based on knowledge and not on numbers. -- Plato > > > > If it's in the news, don't worry about it. The very definition of > > "news" is "something that hardly ever happens." -- Bruce Schneier > > _______________________________________________ > > dm-crypt mailing list > > dm-crypt@xxxxxxxx > > http://www.saout.de/mailman/listinfo/dm-crypt > > > > > _______________________________________________ > dm-crypt mailing list > dm-crypt@xxxxxxxx > http://www.saout.de/mailman/listinfo/dm-crypt -- Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@xxxxxxxxxxx GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718 ---- A good decision is based on knowledge and not on numbers. -- Plato If it's in the news, don't worry about it. The very definition of "news" is "something that hardly ever happens." -- Bruce Schneier _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx http://www.saout.de/mailman/listinfo/dm-crypt
