New OSD Nodes, pgs haven't changed state

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

Yesterday morning I added two more OSD nodes and changed the crushmap from disk to node. It looked to me like everything went ok besides some disks missing that I can re-add later, but the cluster status hasn't changed since then.  Here is the output of ceph -w:

    cluster 395fb046-0062-4252-914c-013258c5575c
     health HEALTH_ERR
            1761 pgs are stuck inactive for more than 300 seconds
            1761 pgs peering
            1761 pgs stuck inactive
            8 requests are blocked > 32 sec
            crush map has legacy tunables (require bobtail, min is firefly)
     monmap e2: 3 mons at {birkeland=192.168.10.190:6789/0,immanuel=192.168.10.1                                     25:6789/0,peratt=192.168.10.187:6789/0}
            election epoch 14, quorum 0,1,2 immanuel,peratt,birkeland
     osdmap e186: 26 osds: 26 up, 26 in; 1796 remapped pgs
            flags sortbitwise
      pgmap v6599413: 1796 pgs, 4 pools, 1343 GB data, 336 kobjects
            4049 GB used, 92779 GB / 96829 GB avail
                1761 remapped+peering
                  35 active+clean
2016-10-09 07:00:00.000776 mon.0 [INF] HEALTH_ERR; 1761 pgs are stuck inactive f                                     or more than 300 seconds; 1761 pgs peering; 1761 pgs stuck inactive; 8 requests                                      are blocked > 32 sec; crush map has legacy tunables (require bobtail, min is fir                                     efly)


I have legacy tunables on since Ceph is only backing our Xenserver infrastructure.  The number of pgs remapping and clean haven't changed and there isn't seem to be that much data... Is this normal behavior?

Here is my crushmap:

# begin crush map
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 1
tunable straw_calc_version 1
# devices
device 0 osd.0
device 1 osd.1
device 2 osd.2
device 3 osd.3
device 4 osd.4
device 5 osd.5
device 6 osd.6
device 7 osd.7
device 8 osd.8
device 9 osd.9
device 10 osd.10
device 11 osd.11
device 12 osd.12
device 13 osd.13
device 14 osd.14
device 15 osd.15
device 16 osd.16
device 17 osd.17
device 18 osd.18
device 19 osd.19
device 20 osd.20
device 21 osd.21
device 22 osd.22
device 23 osd.23
device 24 osd.24
device 25 osd.25
# types
type 0 osd
type 1 host
type 2 chassis
type 3 rack
type 4 row
type 5 pdu
type 6 pod
type 7 room
type 8 datacenter
type 9 region
type 10 root
# buckets
host tesla {
        id -2           # do not change unnecessarily
        # weight 36.369
        alg straw
        hash 0  # rjenkins1
        item osd.5 weight 3.637
        item osd.0 weight 3.637
        item osd.2 weight 3.637
        item osd.4 weight 3.637
        item osd.8 weight 3.637
        item osd.3 weight 3.637
        item osd.6 weight 3.637
        item osd.1 weight 3.637
        item osd.9 weight 3.637
        item osd.7 weight 3.637
}
host faraday {
        id -3           # do not change unnecessarily
        # weight 32.732
        alg straw
        hash 0  # rjenkins1
        item osd.23 weight 3.637
        item osd.18 weight 3.637
        item osd.17 weight 3.637
        item osd.25 weight 3.637
        item osd.20 weight 3.637
        item osd.22 weight 3.637
        item osd.21 weight 3.637
        item osd.19 weight 3.637
        item osd.24 weight 3.637
}
host hertz {
        id -4           # do not change unnecessarily
        # weight 25.458
        alg straw
        hash 0  # rjenkins1
        item osd.15 weight 3.637
        item osd.12 weight 3.637
        item osd.13 weight 3.637
        item osd.14 weight 3.637
        item osd.16 weight 3.637
        item osd.10 weight 3.637
        item osd.11 weight 3.637
}
root default {
        id -1           # do not change unnecessarily
        # weight 94.559
        alg straw
        hash 0  # rjenkins1
        item tesla weight 36.369
        item faraday weight 32.732
        item hertz weight 25.458
}
# rules
rule replicated_ruleset {
        ruleset 0
        type replicated
        min_size 1
        max_size 10
        step take default
        step chooseleaf firstn 0 type host
        step emit
}
# end crush map


Cheers,
Mike
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux