ruleset 3 is:
rule iscsi {
ruleset 3
type replicated
min_size 1
max_size 10
step take iscsi
step chooseleaf firstn 0 type datacenter
step chooseleaf firstn 0 type host
step emit
}
2013/7/16 Vladislav Gorbunov <vadikgo@xxxxxxxxx>:
> sorry, after i'm try to apply crush ruleset 3 (iscsi) to
> pool iscsi:
> ceph osd pool set iscsi crush_ruleset 3
>
> 2013/7/16 Vladislav Gorbunov <vadikgo@xxxxxxxxx>:
>>>Have you run this crush map through any test mappings yet?
>> Yes, it worked on test cluster, and after apply map to main cluster.
>> OSD servers downed after i'm try to apply crush ruleset 3 (iscsi) to
>> pool iscsi:
>> ceph osd pool set data crush_ruleset 3
>>
>> 2013/7/16 Gregory Farnum <greg@xxxxxxxxxxx>:
>>> It's probably not the same issue as that ticket, which was about the
>>> OSD handling a lack of output incorrectly. (It might be handling the
>>> output incorrectly in some other way, but hopefully not...)
>>>
>>> Have you run this crush map through any test mappings yet?
>>> -Greg
>>> Software Engineer #42 @ http://inktank.com | http://ceph.com
>>>
>>>
>>> On Sun, Jul 14, 2013 at 10:59 PM, Vladislav Gorbunov <vadikgo@xxxxxxxxx> wrote:
>>>> Sympthoms like on http://tracker.ceph.com/issues/4699
>>>>
>>>> all OSDs the process ceph-osd crash with segfault
>>>>
>>>> If i stop MONs daemons then i can start OSDs but if i start MONs back
>>>> then die all OSDs again.
>>>>
>>>> more detailed log:
>>>> 0> 2013-07-15 16:42:05.001242 7ffe5a6fc700 -1 *** Caught signal
>>>> (Segmentation fault) **
>>>> in thread 7ffe5a6fc700
>>>>
>>>> ceph version 0.61.4 (1669132fcfc27d0c0b5e5bb93ade59d147e23404)
>>>> 1: /usr/bin/ceph-osd() [0x790e5a]
>>>> 2: (()+0xfcb0) [0x7ffe6b729cb0]
>>>> 3: /usr/bin/ceph-osd() [0x893879]
>>>> 4: (crush_do_rule()+0x1e5) [0x894065]
>>>> 5: (CrushWrapper::do_rule(int, int, std::vector<int,
>>>> std::allocator<int> >&, int, std::vector<unsigned int,
>>>> std::allocator<unsigned int> > const&) const+0x7a) [0x81b2ba]
>>>> 6: (OSDMap::_pg_to_osds(pg_pool_t const&, pg_t, std::vector<int,
>>>> std::allocator<int> >&) const+0x8f) [0x80d7cf]
>>>> 7: (OSDMap::pg_to_up_acting_osds(pg_t, std::vector<int,
>>>> std::allocator<int> >&, std::vector<int, std::allocator<int> >&)
>>>> const+0xa6) [0x80d8a6]
>>>> 8: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&,
>>>> PG::RecoveryCtx*, std::set<boost::intrusive_ptr<PG>,
>>>> std::less<boost::intrusive_ptr<PG> >,
>>>> std::allocator<boost::intrusive_ptr<PG> > >*)+0x190) [0x631b70]
>>>> 9: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*> >
>>>> const&, ThreadPool::TPHandle&)+0x244) [0x632254]
>>>> 10: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> >
>>>> const&, ThreadPool::TPHandle&)+0x12) [0x66d0f2]
>>>> 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0x835a66]
>>>> 12: (ThreadPool::WorkThread::entry()+0x10) [0x837890]
>>>> 13: (()+0x7e9a) [0x7ffe6b721e9a]
>>>> 14: (clone()+0x6d) [0x7ffe69f9fccd]
>>>> NOTE: a copy of the executable, or `objdump -rdS <executable>` is
>>>> needed to interpret this.
>>>>
>>>> --- logging levels ---
>>>> 0/ 5 none
>>>> 0/ 1 lockdep
>>>> 0/ 1 context
>>>> 1/ 1 crush
>>>> 1/ 5 mds
>>>> 1/ 5 mds_balancer
>>>> 1/ 5 mds_locker
>>>> 1/ 5 mds_log
>>>> 1/ 5 mds_log_expire
>>>> 1/ 5 mds_migrator
>>>> 0/ 1 buffer
>>>> 0/ 1 timer
>>>> 0/ 1 filer
>>>> 0/ 1 striper
>>>> 0/ 1 objecter
>>>> 0/ 5 rados
>>>> 0/ 5 rbd
>>>> 0/ 5 journaler
>>>> 0/ 5 objectcacher
>>>> 0/ 5 client
>>>> 0/ 5 osd
>>>> 0/ 5 optracker
>>>> 0/ 5 objclass
>>>> 1/ 3 filestore
>>>> 1/ 3 journal
>>>> 0/ 5 ms
>>>> 1/ 5 mon
>>>> 0/10 monc
>>>> 0/ 5 paxos
>>>> 0/ 5 tp
>>>> 1/ 5 auth
>>>> 1/ 5 crypto
>>>> 1/ 1 finisher
>>>> 1/ 5 heartbeatmap
>>>> 1/ 5 perfcounter
>>>> 1/ 5 rgw
>>>> 1/ 5 hadoop
>>>> 1/ 5 javaclient
>>>> 1/ 5 asok
>>>> 1/ 1 throttle
>>>> -2/-2 (syslog threshold)
>>>> -1/-1 (stderr threshold)
>>>> max_recent 10000
>>>> max_new 1000
>>>> log_file /var/log/ceph/ceph-osd.2.log
>>>> --- end dump of recent events ---
>>>>
>>>> 2013/7/14 Vladislav Gorbunov <vadikgo@xxxxxxxxx>:
>>>>> Hello!
>>>>>
>>>>> After change the crush map all osd (ceph version 0.61.4
>>>>> (1669132fcfc27d0c0b5e5bb93ade59d147e23404)) on pool default is crushed
>>>>> with the error:
>>>>> 2013-07-14 17:26:23.755432 7f0c963ad700 -1 *** Caught signal
>>>>> (Segmentation fault) **
>>>>> in thread 7f0c963ad700
>>>>> ...skipping...
>>>>> 10: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> >
>>>>> const&, ThreadPool::TPHandle&)+0x12) [0x66d0f2]
>>>>> 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0x835a66]
>>>>> 12: (ThreadPool::WorkThread::entry()+0x10) [0x837890]
>>>>> 13: (()+0x7e9a) [0x7fac4e597e9a]
>>>>> 14: (clone()+0x6d) [0x7fac4ce15ccd]
>>>>> NOTE: a copy of the executable, or `objdump -rdS <executable>` is
>>>>> needed to interpret this.
>>>>>
>>>>> --- logging levels ---
>>>>> 0/ 5 none
>>>>> 0/ 1 lockdep
>>>>> 0/ 1 context
>>>>> 1/ 1 crush
>>>>> 1/ 5 mds
>>>>> 1/ 5 mds_balancer
>>>>> 1/ 5 mds_locker
>>>>> 1/ 5 mds_log
>>>>> 1/ 5 mds_log_expire
>>>>> 1/ 5 mds_migrator
>>>>> 0/ 1 buffer
>>>>> 0/ 1 timer
>>>>> 0/ 1 filer
>>>>> 0/ 1 striper
>>>>> 0/ 1 objecter
>>>>> 0/ 5 rados
>>>>> 0/ 5 rbd
>>>>> 0/ 5 journaler
>>>>> 0/ 5 objectcacher
>>>>> 0/ 5 client
>>>>> 0/ 5 osd
>>>>> 0/ 5 optracker
>>>>> 0/ 5 objclass
>>>>> 1/ 3 filestore
>>>>> 1/ 3 journal
>>>>> 0/ 5 ms
>>>>> 1/ 5 mon
>>>>> 0/10 monc
>>>>> 0/ 5 paxos
>>>>> 0/ 5 tp
>>>>> 1/ 5 auth
>>>>> 1/ 5 crypto
>>>>> 1/ 1 finisher
>>>>> 1/ 5 heartbeatmap
>>>>> 1/ 5 perfcounter
>>>>> 1/ 5 rgw
>>>>> 1/ 5 hadoop
>>>>> 1/ 5 javaclient
>>>>> 1/ 5 asok
>>>>> 1/ 1 throttle
>>>>> -2/-2 (syslog threshold)
>>>>> -1/-1 (stderr threshold)
>>>>> max_recent 10000
>>>>> max_new 1000
>>>>> log_file /var/log/ceph/ceph-osd.2.log
>>>>> --- end dump of recent events ---
>>>>>
>>>>> ceph osd complity ignoring osd start/stop an always show this map:
>>>>> # id weight type name up/down reweight
>>>>> -2 65.52 pool iscsi
>>>>> -4 32.76 datacenter datacenter-cod
>>>>> -6 32.76 host gstore1
>>>>> 82 2.73 osd.82 up 1
>>>>> 83 2.73 osd.83 up 1
>>>>> 84 2.73 osd.84 up 1
>>>>> 85 2.73 osd.85 up 1
>>>>> 86 2.73 osd.86 up 1
>>>>> 87 2.73 osd.87 up 1
>>>>> 88 2.73 osd.88 up 1
>>>>> 89 2.73 osd.89 up 1
>>>>> 90 2.73 osd.90 up 1
>>>>> 91 2.73 osd.91 up 1
>>>>> 92 2.73 osd.92 up 1
>>>>> 93 2.73 osd.93 up 1
>>>>> -5 32.76 datacenter datacenter-rcod
>>>>> -7 32.76 host gstore2
>>>>> 94 2.73 osd.94 up 1
>>>>> 95 2.73 osd.95 up 1
>>>>> 96 2.73 osd.96 up 1
>>>>> 97 2.73 osd.97 up 1
>>>>> 98 2.73 osd.98 up 1
>>>>> 99 2.73 osd.99 up 1
>>>>> 100 2.73 osd.100 up 1
>>>>> 101 2.73 osd.101 up 1
>>>>> 102 2.73 osd.102 up 1
>>>>> 103 2.73 osd.103 up 1
>>>>> 104 2.73 osd.104 up 1
>>>>> 105 2.73 osd.105 up 1
>>>>> -1 68.96 pool default
>>>>> -3 68.96 rack unknownrack
>>>>> -9 5 host gstore5
>>>>> 2 5 osd.2 down 1
>>>>> -10 16 host cstore1
>>>>> 5 1 osd.5 down 0
>>>>> 6 1 osd.6 down 0
>>>>> 7 1 osd.7 down 0
>>>>> 8 1 osd.8 down 0
>>>>> 9 1 osd.9 down 0
>>>>> 10 1 osd.10 down 0
>>>>> 11 1 osd.11 down 0
>>>>> 12 1 osd.12 down 0
>>>>> 13 1 osd.13 down 0
>>>>> 14 1 osd.14 down 0
>>>>> 4 1 osd.4 down 0
>>>>> 47 1 osd.47 down 0
>>>>> 48 1 osd.48 down 0
>>>>> 49 1 osd.49 down 0
>>>>> 50 1 osd.50 down 0
>>>>> 51 1 osd.51 down 0
>>>>> -11 21 host cstore2
>>>>> 15 1 osd.15 down 0
>>>>> 16 1 osd.16 down 0
>>>>> 17 1 osd.17 down 0
>>>>> 18 1 osd.18 down 0
>>>>> 19 1 osd.19 down 0
>>>>> 20 1 osd.20 down 0
>>>>> 21 1 osd.21 down 0
>>>>> 22 1 osd.22 down 0
>>>>> 23 1 osd.23 down 0
>>>>> 24 1 osd.24 down 0
>>>>> 41 1 osd.41 down 0
>>>>> 42 1 osd.42 down 0
>>>>> 43 1 osd.43 down 0
>>>>> 44 1 osd.44 down 0
>>>>> 45 1 osd.45 down 0
>>>>> 46 1 osd.46 down 0
>>>>> 52 1 osd.52 down 0
>>>>> 53 1 osd.53 down 0
>>>>> 54 1 osd.54 down 0
>>>>> 55 1 osd.55 down 0
>>>>> 56 1 osd.56 up 1
>>>>> 57 0 osd.57 up 1
>>>>> -12 16 host cstore3
>>>>> 25 1 osd.25 down 0
>>>>> 26 1 osd.26 down 0
>>>>> 27 1 osd.27 down 0
>>>>> 28 1 osd.28 down 0
>>>>> 29 1 osd.29 down 0
>>>>> 30 1 osd.30 down 0
>>>>> 31 1 osd.31 down 0
>>>>> 32 1 osd.32 down 0
>>>>> 33 1 osd.33 down 0
>>>>> 34 1 osd.34 down 0
>>>>> 35 1 osd.35 down 0
>>>>> 36 1 osd.36 down 0
>>>>> 37 1 osd.37 down 0
>>>>> 38 1 osd.38 down 0
>>>>> 39 1 osd.39 down 0
>>>>> 40 1 osd.40 down 0
>>>>> -13 7.64 host cstore4
>>>>> 62 0.55 osd.62 down 0
>>>>> 63 0.55 osd.63 down 0
>>>>> 64 0.55 osd.64 down 0
>>>>> 65 0.55 osd.65 down 0
>>>>> 66 0.55 osd.66 down 0
>>>>> 67 0.55 osd.67 down 0
>>>>> 68 0.55 osd.68 down 0
>>>>> 69 0.55 osd.69 down 0
>>>>> 70 0.27 osd.70 down 0
>>>>> 71 0.27 osd.71 down 0
>>>>> 72 0.27 osd.72 down 0
>>>>> 73 0.27 osd.73 down 0
>>>>> 74 0.27 osd.74 down 0
>>>>> 75 0.27 osd.75 down 0
>>>>> 76 0.27 osd.76 down 0
>>>>> 77 0.27 osd.77 down 0
>>>>> 78 0.27 osd.78 down 0
>>>>> 79 0.27 osd.79 down 0
>>>>> 80 0.27 osd.80 down 0
>>>>> 81 0.27 osd.81 down 0
>>>>> -14 3.32 host cstore5
>>>>> 0 0.4 osd.0 down 0
>>>>> 1 0.5 osd.1 down 0
>>>>> 3 0.4 osd.3 down 0
>>>>> 58 0.4 osd.58 down 1
>>>>> 59 0.54 osd.59 down 1
>>>>> 60 0.54 osd.60 down 1
>>>>> 61 0.54 osd.61 down 1
>>>>>
>>>>> crush map is:
>>>>> # begin crush map
>>>>>
>>>>> # devices
>>>>> device 0 osd.0
>>>>> device 1 osd.1
>>>>> device 2 osd.2
>>>>> device 3 osd.3
>>>>> device 4 osd.4
>>>>> device 5 osd.5
>>>>> device 6 osd.6
>>>>> device 7 osd.7
>>>>> device 8 osd.8
>>>>> device 9 osd.9
>>>>> device 10 osd.10
>>>>> device 11 osd.11
>>>>> device 12 osd.12
>>>>> device 13 osd.13
>>>>> device 14 osd.14
>>>>> device 15 osd.15
>>>>> device 16 osd.16
>>>>> device 17 osd.17
>>>>> device 18 osd.18
>>>>> device 19 osd.19
>>>>> device 20 osd.20
>>>>> device 21 osd.21
>>>>> device 22 osd.22
>>>>> device 23 osd.23
>>>>> device 24 osd.24
>>>>> device 25 osd.25
>>>>> device 26 osd.26
>>>>> device 27 osd.27
>>>>> device 28 osd.28
>>>>> device 29 osd.29
>>>>> device 30 osd.30
>>>>> device 31 osd.31
>>>>> device 32 osd.32
>>>>> device 33 osd.33
>>>>> device 34 osd.34
>>>>> device 35 osd.35
>>>>> device 36 osd.36
>>>>> device 37 osd.37
>>>>> device 38 osd.38
>>>>> device 39 osd.39
>>>>> device 40 osd.40
>>>>> device 41 osd.41
>>>>> device 42 osd.42
>>>>> device 43 osd.43
>>>>> device 44 osd.44
>>>>> device 45 osd.45
>>>>> device 46 osd.46
>>>>> device 47 osd.47
>>>>> device 48 osd.48
>>>>> device 49 osd.49
>>>>> device 50 osd.50
>>>>> device 51 osd.51
>>>>> device 52 osd.52
>>>>> device 53 osd.53
>>>>> device 54 osd.54
>>>>> device 55 osd.55
>>>>> device 56 osd.56
>>>>> device 57 osd.57
>>>>> device 58 osd.58
>>>>> device 59 osd.59
>>>>> device 60 osd.60
>>>>> device 61 osd.61
>>>>> device 62 osd.62
>>>>> device 63 osd.63
>>>>> device 64 osd.64
>>>>> device 65 osd.65
>>>>> device 66 osd.66
>>>>> device 67 osd.67
>>>>> device 68 osd.68
>>>>> device 69 osd.69
>>>>> device 70 osd.70
>>>>> device 71 osd.71
>>>>> device 72 osd.72
>>>>> device 73 osd.73
>>>>> device 74 osd.74
>>>>> device 75 osd.75
>>>>> device 76 osd.76
>>>>> device 77 osd.77
>>>>> device 78 osd.78
>>>>> device 79 osd.79
>>>>> device 80 osd.80
>>>>> device 81 osd.81
>>>>> device 82 osd.82
>>>>> device 83 osd.83
>>>>> device 84 osd.84
>>>>> device 85 osd.85
>>>>> device 86 osd.86
>>>>> device 87 osd.87
>>>>> device 88 osd.88
>>>>> device 89 osd.89
>>>>> device 90 osd.90
>>>>> device 91 osd.91
>>>>> device 92 osd.92
>>>>> device 93 osd.93
>>>>> device 94 osd.94
>>>>> device 95 osd.95
>>>>> device 96 osd.96
>>>>> device 97 osd.97
>>>>> device 98 osd.98
>>>>> device 99 osd.99
>>>>> device 100 osd.100
>>>>> device 101 osd.101
>>>>> device 102 osd.102
>>>>> device 103 osd.103
>>>>> device 104 osd.104
>>>>> device 105 osd.105
>>>>>
>>>>> # types
>>>>> type 0 osd
>>>>> type 1 host
>>>>> type 2 rack
>>>>> type 3 row
>>>>> type 4 room
>>>>> type 5 datacenter
>>>>> type 6 pool
>>>>>
>>>>> # buckets
>>>>> host gstore5 {
>>>>> id -9 # do not change unnecessarily
>>>>> # weight 5.000
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.2 weight 5.000
>>>>> }
>>>>> host cstore1 {
>>>>> id -10 # do not change unnecessarily
>>>>> # weight 16.000
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.5 weight 1.000
>>>>> item osd.6 weight 1.000
>>>>> item osd.7 weight 1.000
>>>>> item osd.8 weight 1.000
>>>>> item osd.9 weight 1.000
>>>>> item osd.10 weight 1.000
>>>>> item osd.11 weight 1.000
>>>>> item osd.12 weight 1.000
>>>>> item osd.13 weight 1.000
>>>>> item osd.14 weight 1.000
>>>>> item osd.4 weight 1.000
>>>>> item osd.47 weight 1.000
>>>>> item osd.48 weight 1.000
>>>>> item osd.49 weight 1.000
>>>>> item osd.50 weight 1.000
>>>>> item osd.51 weight 1.000
>>>>> }
>>>>> host cstore2 {
>>>>> id -11 # do not change unnecessarily
>>>>> # weight 20.000
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.15 weight 1.000
>>>>> item osd.16 weight 1.000
>>>>> item osd.17 weight 1.000
>>>>> item osd.18 weight 1.000
>>>>> item osd.19 weight 1.000
>>>>> item osd.20 weight 1.000
>>>>> item osd.21 weight 1.000
>>>>> item osd.22 weight 1.000
>>>>> item osd.23 weight 1.000
>>>>> item osd.24 weight 1.000
>>>>> item osd.41 weight 1.000
>>>>> item osd.42 weight 1.000
>>>>> item osd.43 weight 1.000
>>>>> item osd.44 weight 1.000
>>>>> item osd.45 weight 1.000
>>>>> item osd.46 weight 1.000
>>>>> item osd.52 weight 1.000
>>>>> item osd.53 weight 1.000
>>>>> item osd.54 weight 1.000
>>>>> item osd.55 weight 1.000
>>>>> item osd.56 weight 0.000
>>>>> item osd.57 weight 0.000
>>>>> }
>>>>> host cstore3 {
>>>>> id -12 # do not change unnecessarily
>>>>> # weight 16.000
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.25 weight 1.000
>>>>> item osd.26 weight 1.000
>>>>> item osd.27 weight 1.000
>>>>> item osd.28 weight 1.000
>>>>> item osd.29 weight 1.000
>>>>> item osd.30 weight 1.000
>>>>> item osd.31 weight 1.000
>>>>> item osd.32 weight 1.000
>>>>> item osd.33 weight 1.000
>>>>> item osd.34 weight 1.000
>>>>> item osd.35 weight 1.000
>>>>> item osd.36 weight 1.000
>>>>> item osd.37 weight 1.000
>>>>> item osd.38 weight 1.000
>>>>> item osd.39 weight 1.000
>>>>> item osd.40 weight 1.000
>>>>> }
>>>>> host cstore4 {
>>>>> id -13 # do not change unnecessarily
>>>>> # weight 7.640
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.62 weight 0.550
>>>>> item osd.63 weight 0.550
>>>>> item osd.64 weight 0.550
>>>>> item osd.65 weight 0.550
>>>>> item osd.66 weight 0.550
>>>>> item osd.67 weight 0.550
>>>>> item osd.68 weight 0.550
>>>>> item osd.69 weight 0.550
>>>>> item osd.70 weight 0.270
>>>>> item osd.71 weight 0.270
>>>>> item osd.72 weight 0.270
>>>>> item osd.73 weight 0.270
>>>>> item osd.74 weight 0.270
>>>>> item osd.75 weight 0.270
>>>>> item osd.76 weight 0.270
>>>>> item osd.77 weight 0.270
>>>>> item osd.78 weight 0.270
>>>>> item osd.79 weight 0.270
>>>>> item osd.80 weight 0.270
>>>>> item osd.81 weight 0.270
>>>>> }
>>>>> host cstore5 {
>>>>> id -14 # do not change unnecessarily
>>>>> # weight 3.320
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.0 weight 0.400
>>>>> item osd.1 weight 0.500
>>>>> item osd.3 weight 0.400
>>>>> item osd.58 weight 0.400
>>>>> item osd.59 weight 0.540
>>>>> item osd.60 weight 0.540
>>>>> item osd.61 weight 0.540
>>>>> }
>>>>> rack unknownrack {
>>>>> id -3 # do not change unnecessarily
>>>>> # weight 67.960
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item gstore5 weight 5.000
>>>>> item cstore1 weight 16.000
>>>>> item cstore2 weight 20.000
>>>>> item cstore3 weight 16.000
>>>>> item cstore4 weight 7.640
>>>>> item cstore5 weight 3.320
>>>>> }
>>>>> pool default {
>>>>> id -1 # do not change unnecessarily
>>>>> # weight 67.960
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item unknownrack weight 67.960
>>>>> }
>>>>> host gstore1 {
>>>>> id -6 # do not change unnecessarily
>>>>> # weight 32.760
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.82 weight 2.730
>>>>> item osd.83 weight 2.730
>>>>> item osd.84 weight 2.730
>>>>> item osd.85 weight 2.730
>>>>> item osd.86 weight 2.730
>>>>> item osd.87 weight 2.730
>>>>> item osd.88 weight 2.730
>>>>> item osd.89 weight 2.730
>>>>> item osd.90 weight 2.730
>>>>> item osd.91 weight 2.730
>>>>> item osd.92 weight 2.730
>>>>> item osd.93 weight 2.730
>>>>> }
>>>>> datacenter datacenter-cod {
>>>>> id -4 # do not change unnecessarily
>>>>> # weight 32.760
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item gstore1 weight 32.760
>>>>> }
>>>>> host gstore2 {
>>>>> id -7 # do not change unnecessarily
>>>>> # weight 32.760
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item osd.94 weight 2.730
>>>>> item osd.95 weight 2.730
>>>>> item osd.96 weight 2.730
>>>>> item osd.97 weight 2.730
>>>>> item osd.98 weight 2.730
>>>>> item osd.99 weight 2.730
>>>>> item osd.100 weight 2.730
>>>>> item osd.101 weight 2.730
>>>>> item osd.102 weight 2.730
>>>>> item osd.103 weight 2.730
>>>>> item osd.104 weight 2.730
>>>>> item osd.105 weight 2.730
>>>>> }
>>>>> datacenter datacenter-rcod {
>>>>> id -5 # do not change unnecessarily
>>>>> # weight 32.760
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item gstore2 weight 32.760
>>>>> }
>>>>> pool iscsi {
>>>>> id -2 # do not change unnecessarily
>>>>> # weight 65.520
>>>>> alg straw
>>>>> hash 0 # rjenkins1
>>>>> item datacenter-cod weight 32.760
>>>>> item datacenter-rcod weight 32.760
>>>>> }
>>>>>
>>>>> # rules
>>>>> rule data {
>>>>> ruleset 0
>>>>> type replicated
>>>>> min_size 1
>>>>> max_size 10
>>>>> step take default
>>>>> step chooseleaf firstn 0 type host
>>>>> step emit
>>>>> }
>>>>> rule metadata {
>>>>> ruleset 1
>>>>> type replicated
>>>>> min_size 1
>>>>> max_size 10
>>>>> step take default
>>>>> step chooseleaf firstn 0 type host
>>>>> step emit
>>>>> }
>>>>> rule rbd {
>>>>> ruleset 2
>>>>> type replicated
>>>>> min_size 1
>>>>> max_size 10
>>>>> step take default
>>>>> step chooseleaf firstn 0 type host
>>>>> step emit
>>>>> }
>>>>>
>>>>> # end crush map
>>>>>
>>>>>
>>>>> Restarting mon don't help.
>>>> _______________________________________________
>>>> ceph-users mailing list
>>>> ceph-users@xxxxxxxxxxxxxx
>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
