On Tue, Dec 15, 2015 at 1:58 AM, Adam Kupczyk <akupczyk@xxxxxxxxxxxx> wrote: > > > On Mon, Dec 14, 2015 at 9:28 PM, Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: >> >> On Mon, Dec 14, 2015 at 5:17 AM, Radoslaw Zarzynski >> <rzarzynski@xxxxxxxxxxxx> wrote: >> > Hello Folks, >> > >> > I would like to publish a proposal regarding improvements to Ceph >> > data-at-rest encryption mechanism. Adam Kupczyk and I worked >> > on that in last weeks. >> > >> > Initially we considered several architectural approaches and made >> > several iterations of discussions with Intel storage group. The proposal >> > is condensed description of the solution we see as the most promising >> > one. >> > >> > We are open to any comments and questions. >> > >> > Regards, >> > Adam Kupczyk >> > Radoslaw Zarzynski >> > >> > >> > ======================= >> > Summary >> > ======================= >> > >> > Data at-rest encryption is mechanism for protecting data center >> > operator from revealing content of physical carriers. >> > >> > Ceph already implements a form of at rest encryption. It is performed >> > through dm-crypt as intermediary layer between OSD and its physical >> > storage. The proposed at rest encryption mechanism will be orthogonal >> > and, in some ways, superior to already existing solution. >> > >> > ======================= >> > Owners >> > ======================= >> > >> > * Radoslaw Zarzynski (Mirantis) >> > * Adam Kupczyk (Mirantis) >> > >> > ======================= >> > Interested Parties >> > ======================= >> > >> > If you are interested in contributing to this blueprint, or want to be >> > a "speaker" during the Summit session, list your name here. >> > >> > Name (Affiliation) >> > Name (Affiliation) >> > Name >> > >> > ======================= >> > Current Status >> > ======================= >> > >> > Current data at rest encryption is achieved through dm-crypt placed >> > under OSD’s filestore. This solution is a generic one and cannot >> > leverage Ceph-specific characteristics. The best example is that >> > encryption is done multiple times - one time for each replica. Another >> > issue is lack of granularity - either OSD encrypts nothing, or OSD >> > encrypts everything (with dm-crypt on). >> > >> > Cryptographic keys are stored on filesystem of storage node that hosts >> > OSDs. Changing them require redeploying the OSDs. >> > >> > The best way to address those issues seems to be introducing >> > encryption into Ceph OSD. >> > >> > ======================= >> > Detailed Description >> > ======================= >> > >> > In addition to the currently available solution, Ceph OSD would >> > accommodate encryption component placed in the replication mechanisms. >> > >> > Data incoming from Ceph clients would be encrypted by primary OSD. It >> > would replicate ciphertext to non-primary members of an acting set. >> > Data sent to Ceph client would be decrypted by OSD handling read >> > operation. This allows to: >> > * perform only one encryption per write, >> > * achieve per-pool key granulation for both key and encryption itself. >> > >> > Unfortunately, having always and everywhere the same key for a given >> > pool is unacceptable - it would make cluster migration and key change >> > extremely burdensome process. To address those issues crypto key >> > versioning would be introduced. All RADOS objects inside single >> > placement group stored on a given OSD would use the same crypto key >> > version. The same PG on other replica may use different version of the >> > same, per pool-granulated key. >> > >> > In typical case ciphertext data transferred from OSD to OSD can be >> > used without change. This is when both OSDs have the same crypto key >> > version for given placement group. In rare cases when crypto keys are >> > different (key change or transition period) receiving OSD will recrypt >> > with local key versions. >> >> I don't understand this part at all. Do you plan to read and rewrite >> the entire PG whenever you change the "key version"? How often do you >> plan to change these keys? What is even the point of changing them, >> since anybody who can control an OSD can grab the entire current key >> set? > > We envision that key change will happen very infrequently. Usually in > reaction to some possible security breach. > After key version is incremented, nothing happens automatically. Old key is > used for as long as PG is not empty. When first RADOS object is created, > the current key version is locked to PG. > There is no solution when someone gets control over OSD - either by running > custom OSD binary or extracting data by impersonating client. It is outside > of scope of at-rest-encryption. We only addressed cases when media storage > somehow leaves datacenter premises. Ability to change key is necessary, > since we need procedure to recover data security after keys are compromised. >> >> >> > For compression to be effective it must be done before encryption. Due >> > to that encryption may be applied differently for replication pools >> > and EC pools. Replicated pools do not implement compression; for those >> > pools encryption is applied right after data enters OSD. For EC pools >> > encryption is applied after compressing. When compression will be >> > implemented for replicated pools, it must be placed before encryption. >> >> So this means you'll be encrypting the object data, but not the omap >> nor xattrs, and not the file names on disk. Is that acceptable to >> people? It's probably fine for a lot of rbd use cases, but not for >> RGW, CephFS, nor raw RADOS where meaningful metadata (and even *data*) >> is stored in those regions. I'd rather a solution worked on the full >> data set. :/ > > We intend to encrypt: > - object data > - omap values > - xattr values > We consider to encrypt: > - object names > - xattr names > We are unable to encrypt: > - omap names Are there any encryption mechanisms that can efficiently and effectively encrypt pieces of data that small? I don't have any expertise in crypto but I thought you needed a certain minimum size of the output blob to get any security at all. If we're turning every 8-byte thing into a 64-byte thing that's going to go much worse for us than just having every OSD encrypt their local disk... -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
