On Wed, Dec 16, 2015 at 11:33 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > On Wed, 16 Dec 2015, Adam Kupczyk wrote: >> On Tue, Dec 15, 2015 at 3:23 PM, Lars Marowsky-Bree <lmb@xxxxxxxx> wrote: >> > On 2015-12-14T14:17:08, Radoslaw Zarzynski <rzarzynski@xxxxxxxxxxxx> wrote: >> > >> > Hi all, >> > >> > great to see this revived. >> > >> > However, I have come to see some concerns with handling the encryption >> > within Ceph itself. >> > >> > The key part to any such approach is formulating the threat scenario. >> > For the use cases we have seen, the data-at-rest encryption matters so >> > they can confidently throw away disks without leaking data. It's not >> > meant as a defense against an online attacker. There usually is no >> > problem with "a few" disks being privileged, or one or two nodes that >> > need an admin intervention for booting (to enter some master encryption >> > key somehow, somewhere). >> > >> > However, that requires *all* data on the OSDs to be encrypted. >> > >> > Crucially, that includes not just the file system meta data (so not just >> > the data), but also the root and especially the swap partition. Those >> > potentially include swapped out data, coredumps, logs, etc. >> > >> > (As an optional feature, it'd be cool if an OSD could be moved to a >> > different chassis and continue operating there, to speed up recovery. >> > Another optional feature would be to eventually be able, for those >> > customers that trust them ;-), supply the key to the on-disk encryption >> > (OPAL et al).) >> > >> > The proposal that Joshua posted a while ago essentially remained based >> > on dm-crypt, but put in simple hooks to retrieve the keys from some >> > "secured" server via sftp/ftps instead of loading them from the root fs. >> > Similar to deo, that ties the key to being on the network and knowing >> > the OSD UUID. >> > >> > This would then also be somewhat easily extensible to utilize the same >> > key management server via initrd/dracut. >> > >> > Yes, this means that each OSD disk is separately encrypted, but given >> > modern CPUs, this is less of a problem. It does have the benefit of >> > being completely transparent to Ceph, and actually covering the whole >> > node. >> Agreed, if encryption is infinitely fast dm-crypt is best solution. >> Below is short analysis of encryption burden for dm-crypt and >> OSD-encryption when using replicated pools. >> >> Summary: >> OSD encryption requires 2.6 times less crypto operations then dm-crypt. > > Yeah, I believe that, but > >> Crypto ops are bottleneck. > > is this really true? I don't think we've tried to measure performance > with dm-crypt, but I also have never heard anyone complain about the > additional CPU utilization or performance impact. Have you observed this? I made tests, mostly on my i7-4910MQ 2.9GHz(4 cores) with SSD. The results for write were appallingly low, I guess due to kernel problems with multi-cpu kcrypto[1]. I will not mention them, these results will obfuscate discussion. And newer kernels >4.0.2 do fixes the issue. The results for read were 350MB/s, but CPU utilization was 44% in kcrypto kernel worker(single core). This effectively means 11 % of total crypto capacity, because intel-optimized AES-NI instruction is used almost every cycle, making hyperthreading useless. [1] http://unix.stackexchange.com/questions/203677/abysmal-general-dm-crypt-luks-write-performance > > sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
