On Fri, Apr 6, 2018 at 10:27 PM, Jeffrey Zhang <zhang.lei.fly+ceph-users@xxxxxxxxx> wrote: > Yes, I am using ceph-volume. > > And i found where the keyring comes from. > > bluestore will save all the information at the starting of disk > (BDEV_LABEL_BLOCK_SIZE=4096) > this area is used for saving labels, including keyring, whoami etc. Correct, this is documented here: http://docs.ceph.com/docs/master/ceph-volume/lvm/activate/#summary (see step 4): Recreate all the files needed with ceph-bluestore-tool prime-osd-dir by pointing it to the OSD block device. > > these can be read through ceph-bluestore-tool show-lable > > $ ceph-bluestore-tool show-label --path /var/lib/ceph/osd/ceph-0 > { > "/var/lib/ceph/osd/ceph-0/block": { > "osd_uuid": "c349b2ba-690f-4a36-b6f6-2cc0d0839f29", > "size": 2147483648, > "btime": "2018-04-04 10:22:25.216117", > "description": "main", > "bluefs": "1", > "ceph_fsid": "14941be9-c327-4a17-8b86-be50ee2f962e", > "kv_backend": "rocksdb", > "magic": "ceph osd volume v026", > "mkfs_done": "yes", > "osd_key": "AQDgNsRaVtsRIBAA6pmOf7y2GBufyE83nHwVvg==", > "ready": "ready", > "whoami": "0" > } > } > > So during mounting the /var/lib/ceph/osd/ceph-0, ceph will dump these > content into the > tmpfs folder. > > On Fri, Apr 6, 2018 at 10:21 PM, David Turner <drakonstein@xxxxxxxxx> wrote: >> >> Likely the differences you're seeing of /dev/sdb1 and tmpfs have to do >> with how ceph-disk vs ceph-volume manage the OSDs and what their defaults >> are. ceph-disk will create partitions on devices while ceph-volume >> configures LVM on the block device. Also with bluestore you do not have a >> standard filesystem, so ceph-volume creates a mock folder to place the >> necessary information into /var/lib/ceph/osd/ceph-0 to track the information >> for the OSD and how to start it. >> >> On Wed, Apr 4, 2018 at 6:20 PM Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: >>> >>> On Tue, Apr 3, 2018 at 6:30 PM Jeffrey Zhang >>> <zhang.lei.fly+ceph-users@xxxxxxxxx> wrote: >>>> >>>> I am testing ceph Luminous, the environment is >>>> >>>> - centos 7.4 >>>> - ceph luminous ( ceph offical repo) >>>> - ceph-deploy 2.0 >>>> - bluestore + separate wal and db >>>> >>>> I found the ceph osd folder `/var/lib/ceph/osd/ceph-0` is mounted >>>> from tmpfs. But where the files in that folder come from? like >>>> `keyring`, >>>> `whoami`? >>> >>> >>> These are generated as part of the initialization process. I don't know >>> the exact commands involved, but the keyring for instance will draw from the >>> results of "ceph osd new" (which is invoked by one of the ceph-volume setup >>> commands). That and whoami are part of the basic information an OSD needs to >>> communicate with a monitor. >>> -Greg >>> >>>> >>>> >>>> $ ls -alh /var/lib/ceph/osd/ceph-0/ >>>> lrwxrwxrwx. 1 ceph ceph 24 Apr 3 16:49 block -> >>>> /dev/ceph-pool/osd0.data >>>> lrwxrwxrwx. 1 root root 22 Apr 3 16:49 block.db -> >>>> /dev/ceph-pool/osd0-db >>>> lrwxrwxrwx. 1 root root 23 Apr 3 16:49 block.wal -> >>>> /dev/ceph-pool/osd0-wal >>>> -rw-------. 1 ceph ceph 37 Apr 3 16:49 ceph_fsid >>>> -rw-------. 1 ceph ceph 37 Apr 3 16:49 fsid >>>> -rw-------. 1 ceph ceph 55 Apr 3 16:49 keyring >>>> -rw-------. 1 ceph ceph 6 Apr 3 16:49 ready >>>> -rw-------. 1 ceph ceph 10 Apr 3 16:49 type >>>> -rw-------. 1 ceph ceph 2 Apr 3 16:49 whoami >>>> >>>> I guess they may be loaded from bluestore. But I can not find any clue >>>> for this. >>>> _______________________________________________ >>>> ceph-users mailing list >>>> ceph-users@xxxxxxxxxxxxxx >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |