On Tue, Jan 9, 2018 at 3:27 PM, Reed Dier <reed.dier@xxxxxxxxxxx> wrote: > After removing the —osd-id flag, everything came up normally. We just verified this is a bug when using --osd-id and that ID is no longer available in the cluster. I've created http://tracker.ceph.com/issues/22642 to get this fixed properly. > > -2 21.82448 host node24 > 0 hdd 7.28450 osd.0 up 1.00000 > 1.00000 > 8 hdd 7.26999 osd.8 up 1.00000 > 1.00000 > 16 hdd 7.26999 osd.16 up 1.00000 > 1.00000 > > > Given the vanilla-ness to this ceph-volume command, is this something > ceph-deploy-able? > > I’m seeing ceph-deploy 1.5.39 as the latest stable release. > 1.5.39 is going to be the last release that supports ceph-disk. We are planning on releasing 2.0.0 (which breaks backwards compatibility) for later this week or early next week. > ceph-deploy --username root disk zap $NODE:$HDD > > ceph-deploy --username root osd create $NODE:$HDD:$SSD > > > In that example $HDD is the main OSD device, and $SSD is the NVMe partition > I want to use for block.db (and block.wal). Or is the syntax different from > the filestore days? The upcoming version does not support re-using an OSD ID, but that might be easy to add. The docs in master are already displaying how the new API looks: http://docs.ceph.com/ceph-deploy/docs/#deploying-osds > And I am assuming that no --bluestore would be necessary given that I am > reading that bluestore is the default and filestore requires intervention. bluestore is the default, correct. The API will change a lot, you might want to > > Thanks, > > Reed > > On Jan 9, 2018, at 2:10 PM, Reed Dier <reed.dier@xxxxxxxxxxx> wrote: > > -2 21.81000 host node24 > 0 hdd 7.26999 osd.0 destroyed 0 > 1.00000 > 8 hdd 7.26999 osd.8 up 1.00000 > 1.00000 > 16 hdd 7.26999 osd.16 up 1.00000 > 1.00000 > > > Should I do these prior to running without the osd-id specified? > > # ceph osd crush remove osd.$ID > # ceph auth del osd.$ID > # ceph osd rm osd.$ID > > > And then it fill in the missing osd.0. > Will set norebalance flag first to prevent data reshuffle upon the osd being > removed from the crush map. > > Thanks, > > Reed > > On Jan 9, 2018, at 2:05 PM, Alfredo Deza <adeza@xxxxxxxxxx> wrote: > > On Tue, Jan 9, 2018 at 2:19 PM, Reed Dier <reed.dier@xxxxxxxxxxx> wrote: > > Hi ceph-users, > > Hoping that this is something small that I am overlooking, but could use the > group mind to help. > > Ceph 12.2.2, Ubuntu 16.04 environment. > OSD (0) is an 8TB spinner (/dev/sda) and I am moving from a filestore > journal to a blocks.db and WAL device on an NVMe partition (/dev/nvme0n1p5). > > I have an OSD that I am trying to convert to bluestore and running into some > trouble. > > Started here until the ceps-volume create statement, which doesn’t work. > http://docs.ceph.com/docs/master/rados/operations/bluestore-migration/ > Worth mentioning I also flushed the journal on the nvme partition before > nuking the OSD. > > $ sudo ceph-osd -i 0 --flush-journal > > > So I first started with this command: > > $ sudo ceph-volume lvm create --bluestore --data /dev/sda --block.db > /dev/nvme0n1p5 --osd-id 0 > > > Pastebin to the ceph-volume log: https://pastebin.com/epkM3aP6 > > However the OSD doesn’t start. > > > I was just able to replicate this by using an ID that doesn't exist in > the cluster. On a cluster with just one OSD (with an ID of 0) I > created > an OSD with --osd-id 3, and had the exact same results. > > > Pastebin to ceph-osd log: https://pastebin.com/9qEsAJzA > > I tried restarting the process, by deleting the LVM structures, zapping the > disk using ceph-volume. > This time using prepare and activate instead of create. > > $ sudo ceph-volume lvm prepare --bluestore --data /dev/sda --block.db > /dev/nvme0n1p5 --osd-id 0 > > $ sudo ceph-volume lvm activate --bluestore 0 > 227e1721-cd2e-4d7e-bb48-bc2bb715a038 > > > Also ran the enable on the ceph-volume systemd unit per > http://docs.ceph.com/docs/master/install/manual-deployment/ > > $ sudo systemctl enable > ceph-volume@lvm-0-227e1721-cd2e-4d7e-bb48-bc2bb715a038 > > > Same results. > > Any help is greatly appreciated. > > > Could you try without passing --osd-id ? > > > Thanks, > > Reed > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |