On Wed, 23 Jan 2019 16:32:08 +0100 Manuel Lausch <manuel.lausch@xxxxxxxx> wrote: > > > > The key api for encryption is *very* odd and a lot of its quirks are > > undocumented. For example, ceph-volume is stuck supporting naming > > files and keys 'lockbox' > > (for backwards compatibility) but there is no real lockbox anymore. > > Another quirk is that when storing the secret in the monitor, it is > > done using the following convention: > > > > dm-crypt/osd/{OSD FSID}/luks > > > > The 'luks' part there doesn't indicate anything about the type of > > encryption (!!) so regardless of the type of encryption (luks or > > plain) the key would still go there. > > > > If you manage to get the keys into the monitors you still wouldn't > > be able to scan OSDs to produce the JSON files, but you would be > > able to create the JSON file with the > > metadata that ceph-volume needs to run the OSD. > > I think it is not that problem to create the json files by myself. > Moving the Keys to the monitors and creating appropriate auth-keys > should be more or less easy as well. > > The problem I see is, that there are individual keys for the journal > and data partition while the new process useses only one key for both > partitions. > > maybe I can recreate the journal partition with the other key. But is > this possible? Are there important data ramaining on the journal after > clean stopping the OSD which I cannot throw away without trashing the > whole OSD? > Ok with a new empty journal the OSD will not start. I have now rescued the data with dd and the recrypt it with a other key and copied the data back. This worked so far Now I encoded the key with base64 and put it to the key-value store. Also created the neccessary authkeys. Creating the json File by hand was quiet easy. But now there is one problem. ceph-disk opens the crypt like cryptsetup --key-file /etc/ceph/dmcrypt-keys/foobar ... ceph-volume pipes the key via stdin like this cat foobar | cryptsetup --key-file - ... The big problem. if the key is given via stdin cryptsetup hashes this key per default with some hash. Only if I set --hash plain it works. I think this is a bug in ceph-volume. Can someone confirm this? there is the related code I mean in ceph-volume https://github.com/ceph/ceph/blob/v12.2.10/src/ceph-volume/ceph_volume/util/encryption.py#L59 Regards Manuel _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com