Hi,
This is ceph.conf during the cluster deploy. ceph version is mimic.
osd pool default size = 3
osd pool default min size = 1
osd pool default pg num = 1024
osd pool default pgp num = 1024
osd crush chooseleaf type = 1
mon_max_pg_per_osd = 2048
mon_allow_pool_delete = true
mon_pg_warn_min_per_osd = 0
mon_pg_warn_max_per_osd = 0
osd_max_pg_per_osd_hard_ratio = 8
Currently we have two class.
# ceph osd crush class ls
[
"hdd",
"ssd"
]
for hdd min_size=1 and replica size=3
for ssd min_size=1 and replica size=2
For HDD:
# ceph osd crush rule dump replicated_rule
{
"rule_id": 0,
"rule_name": "replicated_rule",
"ruleset": 0,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{
"op": "take",
"item": -2,
"item_name": "default~hdd"
},
{
"op": "chooseleaf_firstn",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
}
For SSD:
# ceph osd crush rule dump replicated_rule_ssd
{
"rule_id": 1,
"rule_name": "replicated_rule_ssd",
"ruleset": 1,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{
"op": "take",
"item": -21,
"item_name": "default~ssd"
},
{
"op": "chooseleaf_firstn",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
}
Please suggest me to correct the configuration
Regards,
Munna
On Thu, Dec 9, 2021 at 4:09 PM Stefan Kooman <stefan@xxxxxx> wrote:
> On 12/9/21 09:30, Md. Hejbul Tawhid MUNNA wrote:
> > Hi,
> >
> > Yes, min_size=1 and size=2 for ssd
> >
> > for hdd it is min_size=1 and size=3
> >
> > Could you please advice, about using hdd and ssd in a same ceph cluster.
> > Is it okay for production grade openstack?
>
> Sure. But you want min_size=2 and size=3 for both pools, really. Search
> for discussions about that on the ceph user mailing list (and why you
> would want that for production).
>
> > We have created a new replicated rule for ssd, different pool for ssd
> > and new disk marking ssd class.
> >
> > not idea about ceph-balancer
>
> That probably needs fixing;-). I would start with reading this:
> https://docs.ceph.com/en/latest/rados/operations/balancer/
>
> Check if you have it enabled. If not test what "ceph balancer eval"
> would advise. Enable it and see how it would impove things. Ideally you
> would want to use mode upmap (moving PGs intead of reweighting OSDs).
> In this state the balancer might not want to make any adjustments and
> you might need to change the osd_full_ratio and osd_backfillfull_ratio.
> But the tools I listed in my previous email might even work without
> that. You don't want to put your cluster over the edge (you are already
> close to the edge).
>
> The amount of PGs per OSD seem *really* high. The Ceph Storage Cluster
> has a default maximum value of 300 placement groups per OSD. Have you
> tuned mon_max_pg_per_osd?
>
> Gr. Stefan
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
