On Mon, Feb 4, 2019 at 4:43 AM Hector Martin <hector@xxxxxxxxxxxxxx> wrote: > > On 02/02/2019 05:07, Stuart Longland wrote: > > On 1/2/19 10:43 pm, Alfredo Deza wrote: > >>> The tmpfs setup is expected. All persistent data for bluestore OSDs > >>> setup with LVM are stored in LVM metadata. The LVM/udev handler for > >>> bluestore volumes create these tmpfs filesystems on the fly and populate > >>> them with the information from the metadata. > >> That is mostly what happens. There isn't a dependency on UDEV anymore > >> (yay), but the reason why files are mounted on tmpfs > >> is because *bluestore* spits them out on activation, this makes the > >> path fully ephemeral (a great thing!) > >> > >> The step-by-step is documented in this summary section of 'activate' > >> http://docs.ceph.com/docs/master/ceph-volume/lvm/activate/#summary > >> > >> Filestore doesn't have any of these capabilities and it is why it does > >> have an actual existing path (vs. tmpfs), and the files come from the > >> data partition that > >> gets mounted. > >> > > > > Well, for whatever reason, ceph-osd isn't calling the activate script > > before it starts up. > > > > It is worth noting that the systems I'm using do not use systemd out of > > simplicity. I might need to write an init script to do that. It wasn't > > clear last weekend what commands I needed to run to activate a BlueStore > > OSD. > > The way you do this on Gentoo is by writing the OSD FSID into > /etc/conf.d/ceph-osd.<id>. You need to make note of the ID when the OSD > is first deployed. > > # echo "bluestore_osd_fsid=$(cat /var/lib/ceph/osd/ceph-0/fsid)" > > /etc/conf.d/ceph-osd.0 > > And then of course do the usual initscript symlink enable on Gentoo: > > # ln -s ceph /etc/init.d/ceph-osd.0 > # rc-update add ceph-osd.0 default > > This will then call `ceph-volume lvm activate` for that OSD for you > before bringing it up, which will populate the tmpfs. It is the Gentoo > OpenRC equivalent of enabling the systemd unit for that osd-fsid on > systemd systems (but ceph-volume won't do it for you). This is spot on. The whole systemd script for ceph-volume, is merely passing the ID and FSID over to ceph-volume, which ends up doing something like: ceph-volume lvm activate ID FSID > > You may also want to add some dependencies for all OSDs depending on > your setup (e.g. I run single-host and the mon has the OSD dm-crypt > keys, so that has to come first): > > # echo 'rc_need="ceph-mon.0"' > /etc/conf.d/ceph-osd > > The Gentoo initscript setup for Ceph is unfortunately not very well > documented. I've been meaning to write a blogpost about this to try to > share what I've learned :-) > > -- > Hector Martin (hector@xxxxxxxxxxxxxx) > Public Key: https://marcan.st/marcan.asc > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |