When I create a new encrypted osd with ceph volume[1] I assume something like this is being done, please correct what is wrong. - it creates the pv on the block device - it creates the ceph vg on the block device - it creates the osd lv in the vg - it uses cryptsetup to encrypt this lv (or is there some internal support for luks in lvm?) - it sets all the tags on the vg (shown by: lvs -o lv_tags vg) - it creates and enables ceph-volume@lvm-osdid-osdfsid - it creates and enables ceph-osd@osdid When a node is restarted, these lvm osds are started with - running ceph-volume@lvm-osdid-osdfsid (creating this tmpfs mount?) - running ceph-osd@osdid Q1: I had to create bootstrap-osd/ceph.keyring (ownership root.root). For what is that being used? Does it need to exist upon node restart? Q2: I had some issues with a node starting, solving this with adding a nofail to the fstab. How is this done with ceph-volume? Q3: Why these strange permissions on the mounted folder? drwxrwxrwt 2 ceph ceph 340 Sep 19 15:24 ceph-40 Q4: Where is this luks passphrase stored? Q5: Where does this tmpfs+content come from? How can I mount this myself from the command line? Q6: My lvm tags show ceph.crush_device_class=None, while ceph osd tree shows the correct class. Is this correct? Q7: I saw in my ceph-volume output sometimes 'disabling cephx', what does this mean? How can I verify this and fix it? Links to manuals are also welcome, these ceph-volume[2] are not to clear about this. [1] ceph-volume lvm create --data /dev/sdk --dmcrypt [2] https://docs.ceph.com/en/latest/ceph-volume/lvm/activate/
