That was it, pool 3 had a rule configured that I had removed. So the rule no longer existed.
I hadn't used the pool for some time, it was just used for some testing. I set the rule set to a real one and the cluster is rebalancing as I am typing.
I'll probably delete the pool once I am sure that I have fixed it.
Darryl
________________________________________
From: ceph-users-bounces@xxxxxxxxxxxxxx [ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Darryl Bond [dbond@xxxxxxxxxxxxx]
Sent: Tuesday, April 16, 2013 9:43 AM
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re: Upgrade stale PG
On 04/16/13 08:50, Dan Mick wrote:
>
> On 04/04/2013 02:27 PM, Darryl Bond wrote:
>> # ceph pg 3.8 query
>> pgid currently maps to no osd
> That means your CRUSH rules are wrong. What's the crushmap look like,
> and what's the rule for pool 3?
# begin crush map
# devices
device 0 device0
device 1 device1
device 2 device2
device 3 device3
device 4 device4
device 5 device5
device 6 device6
device 7 device7
device 8 device8
device 9 device9
device 10 osd.10
device 11 osd.11
device 12 osd.12
device 13 osd.13
device 14 osd.14
device 15 osd.15
device 16 device16
device 17 device17
device 18 device18
device 19 device19
device 20 osd.20
device 21 osd.21
device 22 osd.22
device 23 osd.23
device 24 osd.24
device 25 osd.25
device 26 device26
device 27 device27
device 28 device28
device 29 device29
device 30 osd.30
device 31 osd.31
device 32 osd.32
device 33 osd.33
device 34 osd.34
device 35 osd.35
# types
type 0 osd
type 1 host
type 2 rack
type 3 row
type 4 room
type 5 datacenter
type 6 root
# buckets
host ceph1-ssd {
id -2 # do not change unnecessarily
# weight 1.000
alg straw
hash 0 # rjenkins1
item osd.10 weight 0.500
item osd.11 weight 0.500
}
rack ServerRoom-ssd {
id -3 # do not change unnecessarily
# weight 1.000
alg straw
hash 0 # rjenkins1
item ceph1-ssd weight 1.000
}
host ceph2-ssd {
id -4 # do not change unnecessarily
# weight 1.000
alg straw
hash 0 # rjenkins1
item osd.20 weight 0.500
item osd.21 weight 0.500
}
host ceph3-ssd {
id -5 # do not change unnecessarily
# weight 1.000
alg straw
hash 0 # rjenkins1
item osd.30 weight 0.500
item osd.31 weight 0.500
}
rack PABXRoom-ssd {
id -6 # do not change unnecessarily
# weight 1.000
alg straw
hash 0 # rjenkins1
item ceph2-ssd weight 1.000
}
host ceph3-spin {
id -11 # do not change unnecessarily
# weight 16.000
alg straw
hash 0 # rjenkins1
item osd.32 weight 4.000
item osd.33 weight 4.000
item osd.34 weight 4.000
item osd.35 weight 4.000
}
rack BackupCub-spin {
id -7 # do not change unnecessarily
# weight 16.000
alg straw
hash 0 # rjenkins1
item ceph3-spin weight 16.000
}
host ceph1-spin {
id -9 # do not change unnecessarily
# weight 16.000
alg straw
hash 0 # rjenkins1
item osd.12 weight 4.000
item osd.13 weight 4.000
item osd.14 weight 4.000
item osd.15 weight 4.000
}
host ceph2-spin {
id -10 # do not change unnecessarily
# weight 16.000
alg straw
hash 0 # rjenkins1
item osd.22 weight 4.000
item osd.23 weight 4.000
item osd.24 weight 4.000
item osd.25 weight 4.000
}
root spin {
id -8 # do not change unnecessarily
# weight 48.000
alg straw
hash 0 # rjenkins1
item ceph1-spin weight 16.000
item ceph2-spin weight 16.000
item ceph3-spin weight 16.000
}
root ssd {
id -12 # do not change unnecessarily
# weight 3.000
alg straw
hash 0 # rjenkins1
item ceph1-ssd weight 1.000
item ceph2-ssd weight 1.000
item ceph3-ssd weight 1.000
}
rack ServerRoom-spin {
id -13 # do not change unnecessarily
# weight 16.000
alg straw
hash 0 # rjenkins1
item ceph1-spin weight 16.000
}
rack PABXRoom-spin {
id -14 # do not change unnecessarily
# weight 16.000
alg straw
hash 0 # rjenkins1
item ceph2-spin weight 16.000
}
rack BackupCub-ssd {
id -15 # do not change unnecessarily
# weight 1.000
alg straw
hash 0 # rjenkins1
item ceph3-ssd weight 1.000
}
# rules
rule data {
ruleset 0
type replicated
min_size 1
max_size 10
step take spin
step chooseleaf firstn 0 type host
step emit
}
rule metadata {
ruleset 1
type replicated
min_size 1
max_size 10
step take spin
step chooseleaf firstn 0 type host
step emit
}
rule rbd {
ruleset 2
type replicated
min_size 1
max_size 10
step take spin
step chooseleaf firstn 0 type host
step emit
}
rule spin {
ruleset 3
type replicated
min_size 0
max_size 10
step take spin
step chooseleaf firstn 0 type host
step emit
}
rule ssd {
ruleset 4
type replicated
min_size 0
max_size 10
step take ssd
step chooseleaf firstn 0 type host
step emit
}
# end crush map
The contents of this electronic message and any attachments are intended only for the addressee and may contain legally privileged, personal, sensitive or confidential information. If you are not the intended addressee, and have received this email, any transmission, distribution, downloading, printing or photocopying of the contents of this message or attachments is strictly prohibited. Any legal privilege or confidentiality attached to this message and attachments is not waived, lost or destroyed by reason of delivery to any person other than intended addressee. If you have received this message and are not the intended addressee you should notify the sender by return email and destroy all copies of the message and any attachments. Unless expressly attributed, the views expressed in this email do not necessarily represent the views of the company.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
