Re: SSD-primary crush rule doesn't work as intended

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Oh, it's not working as intended though the ssd-primary rule is officially listed on ceph documentation. I should file a feature request or bugzilla for it?

Regards,
Horace Ng


From: "Paul Emmerich" <paul.emmerich@xxxxxxxx>
To: "horace" <horace@xxxxxxxxx>
Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx>
Sent: Wednesday, May 23, 2018 8:37:07 PM
Subject: Re: SSD-primary crush rule doesn't work as intended

You can't mix HDDs and SSDs in a server if you want to use such a rule.
The new selection step after "emit" can't know what server was selected previously.

Paul

2018-05-23 11:02 GMT+02:00 Horace <horace@xxxxxxxxx>:
Add to the info, I have a slightly modified rule to take advantage of the new storage class.

rule ssd-hybrid {
        id 2
        type replicated
        min_size 1
        max_size 10
        step take default class ssd
        step chooseleaf firstn 1 type host
        step emit
        step take default class hdd
        step chooseleaf firstn -1 type host
        step emit
}

Regards,
Horace Ng

----- Original Message -----
From: "horace" <horace@xxxxxxxxx>
To: "ceph-users" <ceph-users@xxxxxxxxxxxxxx>
Sent: Wednesday, May 23, 2018 3:56:20 PM
Subject: SSD-primary crush rule doesn't work as intended

I've set up the rule according to the doc, but some of the PGs are still being assigned to the same host.

http://docs.ceph.com/docs/master/rados/operations/crush-map-edits/

  rule ssd-primary {
              ruleset 5
              type replicated
              min_size 5
              max_size 10
              step take ssd
              step chooseleaf firstn 1 type host
              step emit
              step take platter
              step chooseleaf firstn -1 type host
              step emit
      }

Crush tree:

[root@ceph0 ~]#    ceph osd crush tree
ID CLASS WEIGHT   TYPE NAME     
-1       58.63989 root default   
-2       19.55095     host ceph0
 0   hdd  2.73000         osd.0 
 1   hdd  2.73000         osd.1 
 2   hdd  2.73000         osd.2 
 3   hdd  2.73000         osd.3 
12   hdd  4.54999         osd.12
15   hdd  3.71999         osd.15
18   ssd  0.20000         osd.18
19   ssd  0.16100         osd.19
-3       19.55095     host ceph1
 4   hdd  2.73000         osd.4 
 5   hdd  2.73000         osd.5 
 6   hdd  2.73000         osd.6 
 7   hdd  2.73000         osd.7 
13   hdd  4.54999         osd.13
16   hdd  3.71999         osd.16
20   ssd  0.16100         osd.20
21   ssd  0.20000         osd.21
-4       19.53799     host ceph2
 8   hdd  2.73000         osd.8 
 9   hdd  2.73000         osd.9 
10   hdd  2.73000         osd.10
11   hdd  2.73000         osd.11
14   hdd  3.71999         osd.14
17   hdd  4.54999         osd.17
22   ssd  0.18700         osd.22
23   ssd  0.16100         osd.23

#ceph pg ls-by-pool ssd-hybrid

27.8       1051                  0        0         0       0 4399733760 1581     1581               active+clean 2018-05-23 06:20:56.088216 27957'185553 27959:368828  [23,1,11]         23  [23,1,11]             23 27953'182582 2018-05-23 06:20:56.088172    27843'162478 2018-05-20 18:28:20.118632

With osd.23 and osd.11 being assigned on the same host.

Regards,
Horace Ng
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



--
--
Paul Emmerich

Looking for help with your Ceph cluster? Contact us at https://croit.io

croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux