Re: migrate ceph-disk to ceph-volume fails with dmcrypt

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Jan 24, 2019 at 3:17 PM Manuel Lausch <manuel.lausch@xxxxxxxx> wrote:
>
>
>
> On Wed, 23 Jan 2019 16:32:08 +0100
> Manuel Lausch <manuel.lausch@xxxxxxxx> wrote:
>
>
> > >
> > > The key api for encryption is *very* odd and a lot of its quirks are
> > > undocumented. For example, ceph-volume is stuck supporting naming
> > > files and keys 'lockbox'
> > > (for backwards compatibility) but there is no real lockbox anymore.
> > > Another quirk is that when storing the secret in the monitor, it is
> > > done using the following convention:
> > >
> > >     dm-crypt/osd/{OSD FSID}/luks
> > >
> > > The 'luks' part there doesn't indicate anything about the type of
> > > encryption (!!) so regardless of the type of encryption (luks or
> > > plain) the key would still go there.
> > >
> > > If you manage to get the keys into the monitors you still wouldn't
> > > be able to scan OSDs to produce the JSON files, but you would be
> > > able to create the JSON file with the
> > > metadata that ceph-volume needs to run the OSD.
> >
> > I think it is not that problem to create the json files by myself.
> > Moving the Keys to the monitors and creating appropriate auth-keys
> > should be more or less easy as well.
> >
> > The problem I see is, that there are individual keys for the journal
> > and data partition while the new process useses only one key for both
> > partitions.
> >
> > maybe I can recreate the journal partition with the other key. But is
> > this possible? Are there important data ramaining on the journal after
> > clean stopping the OSD which I cannot throw away without trashing the
> > whole OSD?
> >
>
> Ok with a new empty journal the OSD will not start. I have now rescued
> the data with dd and the recrypt it with a other key and copied the
> data back. This worked so far
>
> Now I encoded the key with base64 and put it to the key-value store.
> Also created the neccessary authkeys. Creating the json File by hand
> was quiet easy.
>
> But now there is one problem.
> ceph-disk opens the crypt like
> cryptsetup --key-file /etc/ceph/dmcrypt-keys/foobar ...
> ceph-volume pipes the key via stdin like this
> cat foobar | cryptsetup --key-file - ...
>
> The big problem. if the key is given via stdin cryptsetup hashes this
> key per default with some hash. Only if I set --hash plain it works. I
> think this is a bug in ceph-volume.
>
> Can someone confirm this?

Ah right, this is when it was supported to have keys in a file.

What type of keys do you have: LUKS or plain?
>
> there is the related code I mean in ceph-volume
> https://github.com/ceph/ceph/blob/v12.2.10/src/ceph-volume/ceph_volume/util/encryption.py#L59
>
> Regards
> Manuel
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux