Re: Ceph extension - how to equilibrate ?

[Peter Maloney <peter.maloney@xxxxxxxxxxxxxxxxxxxx>]

On 04/18/17 16:31, pascal.pucci@xxxxxxxxxxxxxxx wrote:

Hello,

Just an advise : next time, I will extend my Jewel ceph cluster with a fourth node.

Actually, we have 3 x nodes of 12 x OSD with 4TB DD (36 x DD 4TB).

I will add a new node with 12 x 8TB DD (will add 12 new OSD => 48 OSD).

I hope those aren't SMR disks... make sure they're not or it will be very slow, to the point where osds will time out and die.

So, how to simply equilibrate ?

How to just unplug 3 x DD 4TB per node and add to fourth node  and just plug 3 x 8TB in each node coming from fourth node ?

I think you only have to stop them (hopefully not enough to cause missing objects, and optionally set noout first), unmount them, move the disks, mount them and start them on the new node. Then change the crush rule:

ceph osd crush move osd.X host=nodeY

If your journals aren't being moved too, then flush the journals after the osds are stopped:

sync
ceph-osd --id $n --setuser ceph --setgroup ceph --flush-journal

(if that crashes, start the osd, then stop again, and retry)

and before starting them, make new journals.

ceph-osd --id $n --setuser ceph --setgroup ceph --mkjournal

I want at the end : 3 x DD 8TB per node and 9 x DD 4TB per node ?

How to do that in the easyest way ?

I don't want move all data : It will take a long time per OSD...

I don't know how much data this will move if any... but if it moves data, you probably don't have a choice.

Is there a way to just switch OSD between node ?

Thanks for your help.

Pascal,

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com