On Tue, Dec 18, 2018 at 9:11 AM Igor Fedotov <ifedotov@xxxxxxx> wrote:
>
>
> On 12/18/2018 5:07 PM, Sage Weil wrote:
> > On Tue, 18 Dec 2018, Igor Fedotov wrote:
> >> On 12/17/2018 6:23 PM, Alfredo Deza wrote:
> >>> On Mon, Dec 17, 2018 at 9:20 AM Igor Fedotov <ifedotov@xxxxxxx> wrote:
> >>>> Hi All,
> >>>>
> >>>> Got a question while playing with ceph-volume
> >>>>
> >>>> Are --block.db and --block.wal options mandatory for 'ceph-volume lvm
> >>>> prepare --bluestore' command?
> >>> These are not mandatory (per the bluestore spec). Bluestore has 4 ways
> >>> of deploying an OSD:
> >>>
> >>> 1) with a data (block) device only
> >>> 2) with a data device and a block.db
> >>> 3) with a data device and a wal.db
> >>> 4) with a data device, block.db, and wal.db
> >>>
> >>> There is no enforcement for a specific one, any of the above is valid.
> >>>> No formal syntax error is provided if omitted but OSD isn't prepared
> >>>> with the following output:
> >> ...
> >>> Would help here to see what the actual ceph-volume call was (can't
> >>> tell by looking at ceph-osd output)
> >>>
> >>> Looks like you tried several times to deploy for the same OSD? Do you
> >>> think you can provide a way to replicate how you got here?
> >> This is 100% reproducible for fresh "vstart" cluster:
> >>
> >> ../src/vstart -n
> >>
> >> chmod 777 /tmp/<asok-file>
> >>
> >> chmod 777 ./dev
> >>
> >> chmod 777 ./out
> >>
> >> ceph auth add client.bootstrap-osd mon 'allow profile bootstrap-osd'
> >>
> >> ceph auth export -i ./keyring > /var/lib/ceph/bootstrap-osd/ceph.keyring
> >>
> >> CEPH_CONF=ceph.conf /usr/local/bin/ceph-volume lvm prepare --data nvme/data
> >> --bluestore
> > vstart's ceph.conf debugs a bunch of bluestore_* options that normal users
> > generally don't need/use.. maybe that is affecting things?
> >
> > bluestore fsck on mount = true
> > bluestore block create = true
> > bluestore block db path = /home/sage/src/ceph/build/dev/osd$id/block.db.file
> > bluestore block db size = 67108864
> > bluestore block db create = true
> > bluestore block wal path = /home/sage/src/ceph/build/dev/osd$id/block.wal.file
> > bluestore block wal size = 1048576000
> > bluestore block wal create = true
> >
> > sage
>
> Yeah, that's the point. Works fine if these lines are disabled.
Maybe you can try and narrow down what options specifically from that
group are problematic? Without actually trying anything, I can think
that the
paths might be an issue, for example:
bluestore block db path =
/home/sage/src/ceph/build/dev/osd$id/block.db.file
ceph-volume doesn't support any non-standard path like that
>
>
