CRUSH map help

Stephen Mercier <stephen.mercier@xxxxxxxxxxxx> · Thu, 12 May 2016 14:36:33 -0700

I'm trying to setup a crush rule, and I was hoping you guys could clarify something for me.

I have 4 storage nodes across 2 cabinets. (2x2)

I have the crush hierarchy setup to reflect this layout (as follows):

rack cabinet2 {
	id -3		# do not change unnecessarily
	# weight xxxx
	alg straw
	hash 0	# rjenkins1
	item cephstore04 weight xxxx
	item cephstore02 weight xxxx
}
rack cabinet1 {
	id -3		# do not change unnecessarily
	# weight xxxx
	alg straw
	hash 0	# rjenkins1
	item cephstore03 weight xxxx
	item cephstore01 weight xxxx
}
root default {
	id -1		# do not change unnecessarily
	# weight xxxx
	alg straw
	hash 0	# rjenkins1
	item cabinet2 weight xxxx
	item cabinet1 weight xxxx
}

The default ruleset is as follows: (Big surprise!!)

rule replicated_ruleset {
	ruleset 0
	type replicated
	min_size 1
	max_size 10
	step take default
	step choose firstn 0 type osd
	step emit
}

If I want this to ensure that there is at least 1 copy of the data in each cabinet, would I just change it to:

rule replicated_ruleset {
	ruleset 0
	type replicated
	min_size 1
	max_size 10
	step take default
	step choose firstn 0 type rack
	step emit
}

Or should it be:

rule replicated_ruleset {
	ruleset 0
	type replicated
	min_size 1
	max_size 10
	step take default
	step chooseleaf firstn 0 type rack
	step emit
}

Or is there something more complicated I should be doing? I took a look at https://www.mail-archive.com/ceph-users@xxxxxxxxxxxxxx/msg19140.html and it sounds like this is what I want, but I've also see examples like the following:

rule replicated_ruleset {
	ruleset 0
	type replicated
	min_size 1
	max_size 10
	step take default
        step choose firstn 2 type rack
	step chooseleaf firstn 0 type osd
	step emit
}

As you might have noticed, I'm a little confused, so any assistance is greatly appreciated. And just to clarify once more, I want to make sure that it stores at least one copy in each rack. Advice on getting more granular is welcome as well however, as I there are pools with both 2x and 3x replication setup.

Cheers,
-Stephen Mercier | Sr. Systems Architect
Attainia Capital Planning Solutions (ACPS)
O: (650)241-0567, 727 | TF: (866)288-2464, 727
stephen.mercier@xxxxxxxxxxxx | www.attainia.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com