Luminous 12.1.1 rc Our OSD osd.8 failed. So we removed that. We added a new disk and did: $ ceph-deploy osd create --dmcrypt --bluestore store02:/dev/sdd That worked, created osd.18, OSD has data. However, mgr output at http://localhost:7000/servers showed osd.18 under a blank hostname and not e.g. on the node we attached it to. But it is working. "ceph osd tree" looks OK The problem I see is: When I do "ceph osd crush tree" I see the items list under the name:default~hdd tree: device_class:hdd name:store02~hdd type:host but my new drive is missing under this name - there are 5 OSDs, not 6. *However*, if I look further down under the name:default tree device_class:"" name:store02 type:host I see all devices I am expecting, including osd.18 Is this something to worry about? Or is there something needs fixing? Health is warning for scrubbing reasons. Output of related commands below. Thanks for any help, Sean Purdy $ sudo ceph osd tree ID CLASS WEIGHT TYPE NAME UP/DOWN REWEIGHT PRI-AFF -1 32.73651 root default -3 10.91217 host store01 0 hdd 1.81870 osd.0 up 1.00000 1.00000 5 hdd 1.81870 osd.5 up 1.00000 1.00000 6 hdd 1.81870 osd.6 up 1.00000 1.00000 9 hdd 1.81870 osd.9 up 1.00000 1.00000 12 hdd 1.81870 osd.12 up 1.00000 1.00000 15 hdd 1.81870 osd.15 up 1.00000 1.00000 -5 10.91217 host store02 1 hdd 1.81870 osd.1 up 1.00000 1.00000 7 hdd 1.81870 osd.7 up 1.00000 1.00000 10 hdd 1.81870 osd.10 up 1.00000 1.00000 13 hdd 1.81870 osd.13 up 1.00000 1.00000 16 hdd 1.81870 osd.16 up 1.00000 1.00000 18 hdd 1.81870 osd.18 up 1.00000 1.00000 -7 10.91217 host store03 2 hdd 1.81870 osd.2 up 1.00000 1.00000 3 hdd 1.81870 osd.3 up 1.00000 1.00000 4 hdd 1.81870 osd.4 up 1.00000 1.00000 11 hdd 1.81870 osd.11 up 1.00000 1.00000 14 hdd 1.81870 osd.14 up 1.00000 1.00000 17 hdd 1.81870 osd.17 up 1.00000 1.00000 $ sudo ceph osd crush tree [ { "id": -8, "device_class": "hdd", "name": "default~hdd", "type": "root", "type_id": 10, "items": [ { "id": -2, "device_class": "hdd", "name": "store01~hdd", "type": "host", "type_id": 1, "items": [ { "id": 0, "device_class": "hdd", "name": "osd.0", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 5, "device_class": "hdd", "name": "osd.5", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 6, "device_class": "hdd", "name": "osd.6", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 9, "device_class": "hdd", "name": "osd.9", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 12, "device_class": "hdd", "name": "osd.12", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 15, "device_class": "hdd", "name": "osd.15", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 } ] }, { "id": -4, "device_class": "hdd", "name": "store02~hdd", "type": "host", "type_id": 1, "items": [ { "id": 1, "device_class": "hdd", "name": "osd.1", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 7, "device_class": "hdd", "name": "osd.7", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 10, "device_class": "hdd", "name": "osd.10", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 13, "device_class": "hdd", "name": "osd.13", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 16, "device_class": "hdd", "name": "osd.16", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 } ] }, { "id": -6, "device_class": "hdd", "name": "store03~hdd", "type": "host", "type_id": 1, "items": [ { "id": 2, "device_class": "hdd", "name": "osd.2", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 3, "device_class": "hdd", "name": "osd.3", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 4, "device_class": "hdd", "name": "osd.4", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 11, "device_class": "hdd", "name": "osd.11", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 14, "device_class": "hdd", "name": "osd.14", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 17, "device_class": "hdd", "name": "osd.17", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 } ] } ] }, { "id": -1, "device_class": "", "name": "default", "type": "root", "type_id": 10, "items": [ { "id": -3, "device_class": "", "name": "store01", "type": "host", "type_id": 1, "items": [ { "id": 0, "device_class": "hdd", "name": "osd.0", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 5, "device_class": "hdd", "name": "osd.5", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 6, "device_class": "hdd", "name": "osd.6", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 9, "device_class": "hdd", "name": "osd.9", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 12, "device_class": "hdd", "name": "osd.12", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 15, "device_class": "hdd", "name": "osd.15", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 } ] }, { "id": -5, "device_class": "", "name": "store02", "type": "host", "type_id": 1, "items": [ { "id": 1, "device_class": "hdd", "name": "osd.1", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 7, "device_class": "hdd", "name": "osd.7", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 10, "device_class": "hdd", "name": "osd.10", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 13, "device_class": "hdd", "name": "osd.13", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 16, "device_class": "hdd", "name": "osd.16", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 18, "device_class": "hdd", "name": "osd.18", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 } ] }, { "id": -7, "device_class": "", "name": "store03", "type": "host", "type_id": 1, "items": [ { "id": 2, "device_class": "hdd", "name": "osd.2", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 3, "device_class": "hdd", "name": "osd.3", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 4, "device_class": "hdd", "name": "osd.4", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 11, "device_class": "hdd", "name": "osd.11", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 14, "device_class": "hdd", "name": "osd.14", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 }, { "id": 17, "device_class": "hdd", "name": "osd.17", "type": "osd", "type_id": 0, "crush_weight": 1.818695, "depth": 2 } ] } ] } ] $ sudo ceph status cluster: id: a8c34694-a172-4418-a7dd-dd8a642eb545 health: HEALTH_WARN 32 pgs not deep-scrubbed for 86400 174 pgs not scrubbed for 86400 services: mon: 3 daemons, quorum store01,store02,store03 mgr: store01(active) osd: 18 osds: 18 up, 18 in rgw: 3daemonss active data: pools: 6 pools, 672 pgs objects: 4018k objects, 303 GB usage: 1482 GB used, 32040 GB / 33523 GB avail pgs: 672 active+clean io: client: 72099 B/s rd, 542 kB/s wr, 70 op/s rd, 102 op/s wr _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |