Hi
Thank you so much,
Here's the other json file, I'll check and install that and get the logs
asap too. There has not been any snaps on rbd, I haven't used it at all, it
has been just an empty pool.
Br,
Tuomas
-----Original Message-----
From: Sage Weil [mailto:sweil@xxxxxxxxxx]
Sent: 27. huhtikuuta 2015 20:45
To: Tuomas Juntunen
Cc: ceph-users@xxxxxxxxxxxxxx
Subject: Re: Upgrade from Giant to Hammer and after some basic
operations most of the OSD's went down
Yeah, no snaps:
images:
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 17882,
"pool_snaps": [],
"removed_snaps": "[]",
img:
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
...and actually the log shows this happens on pool 2 (rbd), which has
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
I'm guessin gthe offending code is
pi->build_removed_snaps(newly_removed_snaps);
newly_removed_snaps.subtract(cached_removed_snaps);
so newly_removed_snaps should be empty, and apparently cached_removed_snaps
is not? Maybe one of your older osdmaps has snap info for rbd? It doesn't
make sense. :/ Maybe
ceph osd dump 18127 -f json-pretty
just to be certain? I've pushed a branch 'wip-hammer-snaps' that will
appear at gitbuilder.ceph.com in 20-30 minutes that will output some
additional debug info. It will be at
http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/ref/wip-hammer-sanps
or similar, depending on your distro. Can you install it one on node and
start and osd with logging to reproduce the crash?
Thanks!
sage
On Mon, 27 Apr 2015, Tuomas Juntunen wrote:
> Hi
>
> Here you go
>
> Br,
> Tuomas
>
>
>
> -----Original Message-----
> From: Sage Weil [mailto:sweil@xxxxxxxxxx]
> Sent: 27. huhtikuuta 2015 19:23
> To: Tuomas Juntunen
> Cc: 'Samuel Just'; ceph-users@xxxxxxxxxxxxxx
> Subject: Re: Upgrade from Giant to Hammer and after some
> basic operations most of the OSD's went down
>
> On Mon, 27 Apr 2015, Tuomas Juntunen wrote:
> > Thanks for the info.
> >
> > For my knowledge there was no snapshots on that pool, but cannot
> > verify that.
>
> Can you attach a 'ceph osd dump -f json-pretty'? That will shed a bit
> more light on what happened (and the simplest way to fix it).
>
> sage
>
>
> > Any way to make this work again? Removing the tier and other
> > settings didn't fix it, I tried it the second this happened.
> >
> > Br,
> > Tuomas
> >
> > -----Original Message-----
> > From: Samuel Just [mailto:sjust@xxxxxxxxxx]
> > Sent: 27. huhtikuuta 2015 15:50
> > To: tuomas juntunen
> > Cc: ceph-users@xxxxxxxxxxxxxx
> > Subject: Re: Upgrade from Giant to Hammer and after
> > some basic operations most of the OSD's went down
> >
> > So, the base tier is what determines the snapshots for the
> > cache/base pool
> amalgam. You added a populated pool complete with snapshots on top of
> a base tier without snapshots. Apparently, it caused an existential
> crisis for the snapshot code. That's one of the reasons why there is
> a --force-nonempty flag for that operation, I think. I think the
> immediate answer is probably to disallow pools with snapshots as a
> cache tier altogether until we think of a good way to make it work.
> > -Sam
> >
> > ----- Original Message -----
> > From: "tuomas juntunen" <tuomas.juntunen@xxxxxxxxxxxxxxx>
> > To: "Samuel Just" <sjust@xxxxxxxxxx>
> > Cc: ceph-users@xxxxxxxxxxxxxx
> > Sent: Monday, April 27, 2015 4:56:58 AM
> > Subject: Re: Upgrade from Giant to Hammer and after
> > some basic operations most of the OSD's went down
> >
> >
> >
> > The following:
> >
> > ceph osd tier add img images --force-nonempty ceph osd tier
> > cache-mode images forward ceph osd tier set-overlay img images
> >
> > Idea was to make images as a tier to img, move data to img then
> > change
> clients to use the new img pool.
> >
> > Br,
> > Tuomas
> >
> > > Can you explain exactly what you mean by:
> > >
> > > "Also I created one pool for tier to be able to move data without
> outage."
> > >
> > > -Sam
> > > ----- Original Message -----
> > > From: "tuomas juntunen" <tuomas.juntunen@xxxxxxxxxxxxxxx>
> > > To: "Ian Colle" <icolle@xxxxxxxxxx>
> > > Cc: ceph-users@xxxxxxxxxxxxxx
> > > Sent: Monday, April 27, 2015 4:23:44 AM
> > > Subject: Re: Upgrade from Giant to Hammer and after
> > > some basic operations most of the OSD's went down
> > >
> > > Hi
> > >
> > > Any solution for this yet?
> > >
> > > Br,
> > > Tuomas
> > >
> > >> It looks like you may have hit
> > >> http://tracker.ceph.com/issues/7915
> > >>
> > >> Ian R. Colle
> > >> Global Director
> > >> of Software Engineering
> > >> Red Hat (Inktank is now part of Red Hat!)
> > >> http://www.linkedin.com/in/ircolle
> > >> http://www.twitter.com/ircolle
> > >> Cell: +1.303.601.7713
> > >> Email: icolle@xxxxxxxxxx
> > >>
> > >> ----- Original Message -----
> > >> From: "tuomas juntunen" <tuomas.juntunen@xxxxxxxxxxxxxxx>
> > >> To: ceph-users@xxxxxxxxxxxxxx
> > >> Sent: Monday, April 27, 2015 1:56:29 PM
> > >> Subject: Upgrade from Giant to Hammer and after some
> > >> basic operations most of the OSD's went down
> > >>
> > >>
> > >>
> > >> I upgraded Ceph from 0.87 Giant to 0.94.1 Hammer
> > >>
> > >> Then created new pools and deleted some old ones. Also I created
> > >> one pool for tier to be able to move data without outage.
> > >>
> > >> After these operations all but 10 OSD's are down and creating
> > >> this kind of messages to logs, I get more than 100gb of these in a
night:
> > >>
> > >> -19> 2015-04-27 10:17:08.808584 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> > >> les/c
> > >> 16609/16659
> > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> > >> crt=8480'7 lcod
> > >> 0'0 inactive NOTIFY] enter Started
> > >> -18> 2015-04-27 10:17:08.808596 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> > >> les/c
> > >> 16609/16659
> > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> > >> crt=8480'7 lcod
> > >> 0'0 inactive NOTIFY] enter Start
> > >> -17> 2015-04-27 10:17:08.808608 7fd8e748d700 1 osd.23 pg_epoch:
> > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> > >> les/c
> > >> 16609/16659
> > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> > >> crt=8480'7 lcod
> > >> 0'0 inactive NOTIFY] state<Start>: transitioning to Stray
> > >> -16> 2015-04-27 10:17:08.808621 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> > >> les/c
> > >> 16609/16659
> > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> > >> crt=8480'7 lcod
> > >> 0'0 inactive NOTIFY] exit Start 0.000025 0 0.000000
> > >> -15> 2015-04-27 10:17:08.808637 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> > >> les/c
> > >> 16609/16659
> > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> > >> crt=8480'7 lcod
> > >> 0'0 inactive NOTIFY] enter Started/Stray
> > >> -14> 2015-04-27 10:17:08.808796 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> > >> 17879/17879
> > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive
> > >> NOTIFY] exit Reset 0.119467 4 0.000037
> > >> -13> 2015-04-27 10:17:08.808817 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> > >> 17879/17879
> > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive
> > >> NOTIFY] enter Started
> > >> -12> 2015-04-27 10:17:08.808828 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> > >> 17879/17879
> > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive
> > >> NOTIFY] enter Start
> > >> -11> 2015-04-27 10:17:08.808838 7fd8e748d700 1 osd.23 pg_epoch:
> > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> > >> 17879/17879
> > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive
> > >> NOTIFY]
> > >> state<Start>: transitioning to Stray
> > >> -10> 2015-04-27 10:17:08.808849 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> > >> 17879/17879
> > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive
> > >> NOTIFY] exit Start 0.000020 0 0.000000
> > >> -9> 2015-04-27 10:17:08.808861 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> > >> 17879/17879
> > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive
> > >> NOTIFY] enter Started/Stray
> > >> -8> 2015-04-27 10:17:08.809427 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> inactive] exit Reset 7.511623 45 0.000165
> > >> -7> 2015-04-27 10:17:08.809445 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> inactive] enter Started
> > >> -6> 2015-04-27 10:17:08.809456 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> inactive] enter Start
> > >> -5> 2015-04-27 10:17:08.809468 7fd8e748d700 1 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> inactive]
> > >> state<Start>: transitioning to Primary
> > >> -4> 2015-04-27 10:17:08.809479 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> inactive] exit Start 0.000023 0 0.000000
> > >> -3> 2015-04-27 10:17:08.809492 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> inactive] enter Started/Primary
> > >> -2> 2015-04-27 10:17:08.809502 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> inactive] enter Started/Primary/Peering
> > >> -1> 2015-04-27 10:17:08.809513 7fd8e748d700 5 osd.23 pg_epoch:
> > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0
> > >> peering] enter Started/Primary/Peering/GetInfo
> > >> 0> 2015-04-27 10:17:08.813837 7fd8e748d700 -1
> ./include/interval_set.h:
> > >> In
> > >> function 'void interval_set<T>::erase(T, T) [with T = snapid_t]'
> > >> thread
> > >> 7fd8e748d700 time 2015-04-27 10:17:08.809899
> > >> ./include/interval_set.h: 385: FAILED assert(_size >= 0)
> > >>
> > >> ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)
> > >> 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char
> > >> const*)+0x8b)
> > >> [0xbc271b]
> > >> 2: (interval_set<snapid_t>::subtract(interval_set<snapid_t>
> > >> const&)+0xb0) [0x82cd50]
> > >> 3: (PGPool::update(std::tr1::shared_ptr<OSDMap const>)+0x52e)
> > >> [0x80113e]
> > >> 4: (PG::handle_advance_map(std::tr1::shared_ptr<OSDMap const>,
> > >> std::tr1::shared_ptr<OSDMap const>, std::vector<int,
> > >> std::allocator<int> >&, int, std::vector<int, std::allocator<int>
> > >> >&, int, PG::RecoveryCtx*)+0x282) [0x801652]
> > >> 5: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&,
> > >> PG::RecoveryCtx*, std::set<boost::intrusive_ptr<PG>,
> > >> std::less<boost::intrusive_ptr<PG> >,
> > >> std::allocator<boost::intrusive_ptr<PG> > >*)+0x2c3) [0x6b0e43]
> > >> 6: (OSD::process_peering_events(std::list<PG*,
> > >> std::allocator<PG*>
> > >> > const&,
> > >> ThreadPool::TPHandle&)+0x21c) [0x6b191c]
> > >> 7: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*>
> > >> > const&,
> > >> ThreadPool::TPHandle&)+0x18) [0x709278]
> > >> 8: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa5e)
> > >> [0xbb38ae]
> > >> 9: (ThreadPool::WorkThread::entry()+0x10) [0xbb4950]
> > >> 10: (()+0x8182) [0x7fd906946182]
> > >> 11: (clone()+0x6d) [0x7fd904eb147d]
> > >>
> > >> Also by monitoring (ceph -w) I get the following messages, also
> > >> lots of
> them.
> > >>
> > >> 2015-04-27 10:39:52.935812 mon.0 [INF] from='client.?
> 10.20.0.13:0/1174409'
> > >> entity='osd.30' cmd=[{"prefix": "osd crush create-or-move", "args":
> > >> ["host=ceph3", "root=default"], "id": 30, "weight": 1.82}]:
> > >> dispatch
> > >> 2015-04-27 10:39:53.297376 mon.0 [INF] from='client.?
> 10.20.0.13:0/1174483'
> > >> entity='osd.26' cmd=[{"prefix": "osd crush create-or-move", "args":
> > >> ["host=ceph3", "root=default"], "id": 26, "weight": 1.82}]:
> > >> dispatch
> > >>
> > >>
> > >> This is a cluster of 3 nodes with 36 OSD's, nodes are also mons
> > >> and mds's to save servers. All run Ubuntu 14.04.2.
> > >>
> > >> I have pretty much tried everything I could think of.
> > >>
> > >> Restarting daemons doesn't help.
> > >>
> > >> Any help would be appreciated. I can also provide more logs if
> > >> necessary. They just seem to get pretty large in few moments.
> > >>
> > >> Thank you
> > >> Tuomas
> > >>
> > >>
> > >> _______________________________________________
> > >> ceph-users mailing list
> > >> ceph-users@xxxxxxxxxxxxxx
> > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > >>
> > >>
> > >>
> > >
> > >
> > > _______________________________________________
> > > ceph-users mailing list
> > > ceph-users@xxxxxxxxxxxxxx
> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > >
> > >
> > >
> >
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@xxxxxxxxxxxxxx
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> >
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@xxxxxxxxxxxxxx
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> >
>
{
"epoch": 18127,
"fsid": "a2974742-3805-4cd3-bc79-765f2bddaefe",
"created": "2014-10-15 20:43:45.186949",
"modified": "2015-04-27 16:59:43.705499",
"flags": "",
"cluster_snapshot": "",
"pool_max": 17,
"max_osd": 71,
"pools": [
{
"pool": 0,
"pool_name": "data",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 45,
"last_change": "1112",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 1,
"pool_name": "metadata",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "1114",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 2,
"pool_name": "rbd",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 2,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "1116",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 3,
"pool_name": "volumes",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "9974",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 23,
"snap_epoch": 9974,
"pool_snaps": [],
"removed_snaps": "[1~17]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "default",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 4,
"pool_name": "images",
"flags": 9,
"flags_names": "hashpspool,incomplete_clones",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "17905",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 17882,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "default",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 6,
"pool_name": "vms",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "1122",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "default",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 7,
"pool_name": "san",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "14096",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 8,
"pool_name": "vol-ssd-accelerated",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 1024,
"pg_placement_num": 1024,
"crash_replay_interval": 0,
"last_change": "17861",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 14,
"pool_name": "backup",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 128,
"pg_placement_num": 128,
"crash_replay_interval": 0,
"last_change": "18018",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 15,
"pool_name": "img",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 256,
"pg_placement_num": 256,
"crash_replay_interval": 0,
"last_change": "18019",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 16,
"pool_name": "vm",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 1024,
"pg_placement_num": 1024,
"crash_replay_interval": 0,
"last_change": "18020",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 17,
"pool_name": "infradisks",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 256,
"pg_placement_num": 256,
"crash_replay_interval": 0,
"last_change": "18021",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
}
],
"osds": [
{
"osd": 0,
"uuid": "757c3bc5-4d00-4344-8de4-82f5379c96af",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15738,
"last_clean_end": 16079,
"up_from": 16081,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6981\/245797",
"cluster_addr": "10.20.0.11:6881\/5245797",
"heartbeat_back_addr": "10.20.0.11:6883\/5245797",
"heartbeat_front_addr": "10.20.0.11:6887\/5245797",
"state": [
"autoout",
"exists"
]
},
{
"osd": 1,
"uuid": "c7eaa4ac-99fc-46db-84aa-a67274896ec8",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15740,
"last_clean_end": 16079,
"up_from": 16083,
"up_thru": 17873,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6813\/245273",
"cluster_addr": "10.20.0.11:6888\/4245273",
"heartbeat_back_addr": "10.20.0.11:6890\/4245273",
"heartbeat_front_addr": "10.20.0.11:6894\/4245273",
"state": [
"autoout",
"exists"
]
},
{
"osd": 2,
"uuid": "206b2949-4adf-4789-8e06-f68a8ee819c9",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15739,
"last_clean_end": 16079,
"up_from": 16081,
"up_thru": 17874,
"down_at": 17915,
"lost_at": 0,
"public_addr": "10.20.0.11:6808\/245269",
"cluster_addr": "10.20.0.11:6811\/4245269",
"heartbeat_back_addr": "10.20.0.11:6823\/4245269",
"heartbeat_front_addr": "10.20.0.11:6831\/4245269",
"state": [
"autoout",
"exists"
]
},
{
"osd": 3,
"uuid": "90b7c219-4dcd-48ea-a24d-f3b796a521e4",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15736,
"last_clean_end": 16125,
"up_from": 16128,
"up_thru": 17873,
"down_at": 17920,
"lost_at": 0,
"public_addr": "10.20.0.11:6848\/245357",
"cluster_addr": "10.20.0.11:6886\/6245357",
"heartbeat_back_addr": "10.20.0.11:6891\/6245357",
"heartbeat_front_addr": "10.20.0.11:6892\/6245357",
"state": [
"autoout",
"exists"
]
},
{
"osd": 4,
"uuid": "049ef94f-121a-4e71-8ba6-27eaebf0a569",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15737,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17871,
"down_at": 17925,
"lost_at": 0,
"public_addr": "10.20.0.11:6854\/245375",
"cluster_addr": "10.20.0.11:6849\/5245375",
"heartbeat_back_addr": "10.20.0.11:6850\/5245375",
"heartbeat_front_addr": "10.20.0.11:6853\/5245375",
"state": [
"autoout",
"exists"
]
},
{
"osd": 5,
"uuid": "2437a53b-339e-45af-b4de-0fc675d27405",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15734,
"last_clean_end": 16087,
"up_from": 16089,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6834\/245323",
"cluster_addr": "10.20.0.11:6803\/4245323",
"heartbeat_back_addr": "10.20.0.11:6804\/4245323",
"heartbeat_front_addr": "10.20.0.11:6805\/4245323",
"state": [
"autoout",
"exists"
]
},
{
"osd": 6,
"uuid": "f117ceed-b1fd-4069-99fe-b7aba9f3ef8d",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15738,
"last_clean_end": 16079,
"up_from": 16083,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6859\/245401",
"cluster_addr": "10.20.0.11:6865\/5245401",
"heartbeat_back_addr": "10.20.0.11:6868\/5245401",
"heartbeat_front_addr": "10.20.0.11:6874\/5245401",
"state": [
"autoout",
"exists"
]
},
{
"osd": 7,
"uuid": "e98e9b8a-9c62-4e3e-bdb4-c2c30103c0c1",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15730,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17872,
"down_at": 17925,
"lost_at": 0,
"public_addr": "10.20.0.11:6801\/245246",
"cluster_addr": "10.20.0.11:6812\/5245246",
"heartbeat_back_addr": "10.20.0.11:6814\/5245246",
"heartbeat_front_addr": "10.20.0.11:6821\/5245246",
"state": [
"autoout",
"exists"
]
},
{
"osd": 8,
"uuid": "41e471cd-fafe-4422-8bf5-22018bbe1375",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15736,
"last_clean_end": 17783,
"up_from": 17795,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6810\/458698",
"cluster_addr": "10.20.0.11:6819\/458698",
"heartbeat_back_addr": "10.20.0.11:6826\/458698",
"heartbeat_front_addr": "10.20.0.11:6833\/458698",
"state": [
"autoout",
"exists"
]
},
{
"osd": 9,
"uuid": "d68eeebd-d058-4b1c-a30a-994bf8fc8030",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15733,
"last_clean_end": 16079,
"up_from": 16081,
"up_thru": 17873,
"down_at": 17930,
"lost_at": 0,
"public_addr": "10.20.0.11:6969\/245600",
"cluster_addr": "10.20.0.11:6847\/5245600",
"heartbeat_back_addr": "10.20.0.11:6860\/5245600",
"heartbeat_front_addr": "10.20.0.11:6864\/5245600",
"state": [
"autoout",
"exists"
]
},
{
"osd": 10,
"uuid": "660747d6-3f47-449a-bc69-5399b0d54ff6",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15736,
"last_clean_end": 16124,
"up_from": 16128,
"up_thru": 17875,
"down_at": 17935,
"lost_at": 0,
"public_addr": "10.20.0.11:6822\/245281",
"cluster_addr": "10.20.0.11:6855\/4245281",
"heartbeat_back_addr": "10.20.0.11:6856\/4245281",
"heartbeat_front_addr": "10.20.0.11:6866\/4245281",
"state": [
"autoout",
"exists"
]
},
{
"osd": 11,
"uuid": "805965b1-127f-44a6-9a05-8a643eb7a512",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15737,
"last_clean_end": 17786,
"up_from": 17801,
"up_thru": 17870,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6835\/458813",
"cluster_addr": "10.20.0.11:6836\/458813",
"heartbeat_back_addr": "10.20.0.11:6837\/458813",
"heartbeat_front_addr": "10.20.0.11:6838\/458813",
"state": [
"autoout",
"exists"
]
},
{
"osd": 12,
"uuid": "61fbfcbe-d642-478f-9620-f9d72ee96238",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16126,
"up_from": 16129,
"up_thru": 17872,
"down_at": 17935,
"lost_at": 0,
"public_addr": "10.20.0.12:6873\/7542",
"cluster_addr": "10.20.0.12:6947\/4007542",
"heartbeat_back_addr": "10.20.0.12:6948\/4007542",
"heartbeat_front_addr": "10.20.0.12:6949\/4007542",
"state": [
"autoout",
"exists"
]
},
{
"osd": 13,
"uuid": "6faad33b-00be-42a4-92ba-08be5ab7f995",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16705,
"up_from": 16707,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6821\/7274",
"cluster_addr": "10.20.0.12:6859\/5007274",
"heartbeat_back_addr": "10.20.0.12:6860\/5007274",
"heartbeat_front_addr": "10.20.0.12:6861\/5007274",
"state": [
"autoout",
"exists"
]
},
{
"osd": 14,
"uuid": "f301705b-e725-443d-96e1-d9ec9aafe657",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16962,
"up_from": 16966,
"up_thru": 17871,
"down_at": 17940,
"lost_at": 0,
"public_addr": "10.20.0.12:6809\/7209",
"cluster_addr": "10.20.0.12:6803\/7007209",
"heartbeat_back_addr": "10.20.0.12:6804\/7007209",
"heartbeat_front_addr": "10.20.0.12:6810\/7007209",
"state": [
"autoout",
"exists"
]
},
{
"osd": 15,
"uuid": "536bf483-10de-44b0-8e1e-4f349fbe572a",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15785,
"last_clean_end": 16700,
"up_from": 16715,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6881\/7613",
"cluster_addr": "10.20.0.12:6845\/7007613",
"heartbeat_back_addr": "10.20.0.12:6849\/7007613",
"heartbeat_front_addr": "10.20.0.12:6850\/7007613",
"state": [
"autoout",
"exists"
]
},
{
"osd": 16,
"uuid": "4185bd20-8eb0-4616-b36e-bacb181ae40e",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16688,
"up_from": 16693,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6813\/7210",
"cluster_addr": "10.20.0.12:6814\/7007210",
"heartbeat_back_addr": "10.20.0.12:6815\/7007210",
"heartbeat_front_addr": "10.20.0.12:6816\/7007210",
"state": [
"autoout",
"exists"
]
},
{
"osd": 17,
"uuid": "a6f2f5b4-477f-48f9-9acf-d5b7a6c88b98",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6817\/7215",
"cluster_addr": "10.20.0.12:6840\/3007215",
"heartbeat_back_addr": "10.20.0.12:6843\/3007215",
"heartbeat_front_addr": "10.20.0.12:6844\/3007215",
"state": [
"autoout",
"exists"
]
},
{
"osd": 18,
"uuid": "b31b0bd8-938a-496d-91bc-19bf4f794f82",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16700,
"up_from": 16712,
"up_thru": 17872,
"down_at": 17942,
"lost_at": 0,
"public_addr": "10.20.0.12:6805\/7185",
"cluster_addr": "10.20.0.12:6829\/5007185",
"heartbeat_back_addr": "10.20.0.12:6830\/5007185",
"heartbeat_front_addr": "10.20.0.12:6841\/5007185",
"state": [
"autoout",
"exists"
]
},
{
"osd": 19,
"uuid": "d76b6bd5-1ef3-436c-a75d-3587c515eb56",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16693,
"up_from": 16706,
"up_thru": 17872,
"down_at": 16693,
"lost_at": 0,
"public_addr": "10.20.0.12:6885\/7622",
"cluster_addr": "10.20.0.12:6802\/7007622",
"heartbeat_back_addr": "10.20.0.12:6818\/7007622",
"heartbeat_front_addr": "10.20.0.12:6819\/7007622",
"state": [
"exists",
"up"
]
},
{
"osd": 20,
"uuid": "8e4dd982-a4c5-4ca5-9fc5-243f55c4db57",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16703,
"up_from": 16705,
"up_thru": 17872,
"down_at": 16703,
"lost_at": 0,
"public_addr": "10.20.0.12:6825\/7316",
"cluster_addr": "10.20.0.12:6827\/5007316",
"heartbeat_back_addr": "10.20.0.12:6831\/5007316",
"heartbeat_front_addr": "10.20.0.12:6851\/5007316",
"state": [
"exists",
"up"
]
},
{
"osd": 21,
"uuid": "760aaf28-0a34-4bbc-af0c-2654b0a43fff",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16699,
"up_from": 16701,
"up_thru": 17870,
"down_at": 16698,
"lost_at": 0,
"public_addr": "10.20.0.12:6801\/7180",
"cluster_addr": "10.20.0.12:6820\/7007180",
"heartbeat_back_addr": "10.20.0.12:6822\/7007180",
"heartbeat_front_addr": "10.20.0.12:6828\/7007180",
"state": [
"exists",
"up"
]
},
{
"osd": 22,
"uuid": "40322a34-ab31-4760-b71e-a7672f812cb3",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16966,
"up_from": 16969,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6869\/7509",
"cluster_addr": "10.20.0.12:6824\/8007509",
"heartbeat_back_addr": "10.20.0.12:6832\/8007509",
"heartbeat_front_addr": "10.20.0.12:6834\/8007509",
"state": [
"autoout",
"exists"
]
},
{
"osd": 23,
"uuid": "e1d81949-f4b5-4cf2-b6af-dccaaeb30ed7",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17875,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6877\/7592",
"cluster_addr": "10.20.0.12:6826\/4007592",
"heartbeat_back_addr": "10.20.0.12:6835\/4007592",
"heartbeat_front_addr": "10.20.0.12:6839\/4007592",
"state": [
"autoout",
"exists"
]
},
{
"osd": 24,
"uuid": "ede77283-a423-4c6b-9c6e-b0e807c63cb5",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15688,
"last_clean_end": 16109,
"up_from": 16112,
"up_thru": 17871,
"down_at": 16109,
"lost_at": 0,
"public_addr": "10.20.0.13:6921\/213483",
"cluster_addr": "10.20.0.13:6815\/3213483",
"heartbeat_back_addr": "10.20.0.13:6841\/3213483",
"heartbeat_front_addr": "10.20.0.13:6846\/3213483",
"state": [
"exists",
"up"
]
},
{
"osd": 25,
"uuid": "7cfe85f8-3ae9-493d-9801-025ff6c6265d",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15686,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6801\/212863",
"cluster_addr": "10.20.0.13:6918\/3212863",
"heartbeat_back_addr": "10.20.0.13:6932\/3212863",
"heartbeat_front_addr": "10.20.0.13:6934\/3212863",
"state": [
"exists",
"up"
]
},
{
"osd": 26,
"uuid": "266f6d70-519f-4c24-bca2-236495a600a7",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15692,
"last_clean_end": 16125,
"up_from": 16130,
"up_thru": 17872,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6917\/213469",
"cluster_addr": "10.20.0.13:6896\/4213469",
"heartbeat_back_addr": "10.20.0.13:6945\/4213469",
"heartbeat_front_addr": "10.20.0.13:6946\/4213469",
"state": [
"exists",
"up"
]
},
{
"osd": 27,
"uuid": "68644fa9-9459-4db0-a6c9-01661645038b",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15689,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6899\/213448",
"cluster_addr": "10.20.0.13:6876\/3213448",
"heartbeat_back_addr": "10.20.0.13:6902\/3213448",
"heartbeat_front_addr": "10.20.0.13:6904\/3213448",
"state": [
"exists",
"up"
]
},
{
"osd": 28,
"uuid": "fc3d5749-7673-4100-a0d4-f25e9cc0bc88",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15688,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6892\/213419",
"cluster_addr": "10.20.0.13:6866\/5213419",
"heartbeat_back_addr": "10.20.0.13:6867\/5213419",
"heartbeat_front_addr": "10.20.0.13:6873\/5213419",
"state": [
"exists",
"up"
]
},
{
"osd": 29,
"uuid": "cb5feda9-de3f-4e42-bb73-7945b4928b22",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15689,
"last_clean_end": 16109,
"up_from": 16113,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6937\/213527",
"cluster_addr": "10.20.0.13:6898\/2213527",
"heartbeat_back_addr": "10.20.0.13:6926\/2213527",
"heartbeat_front_addr": "10.20.0.13:6927\/2213527",
"state": [
"autoout",
"exists"
]
},
{
"osd": 30,
"uuid": "ef1e65bb-a634-4096-9466-1262af55db01",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15693,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6910\/213454",
"cluster_addr": "10.20.0.13:6823\/4213454",
"heartbeat_back_addr": "10.20.0.13:6838\/4213454",
"heartbeat_front_addr": "10.20.0.13:6852\/4213454",
"state": [
"autoout",
"exists"
]
},
{
"osd": 31,
"uuid": "3dad6393-67a8-43d4-ba8d-ffd320827396",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15688,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6929\/213515",
"cluster_addr": "10.20.0.13:6909\/4213515",
"heartbeat_back_addr": "10.20.0.13:6940\/4213515",
"heartbeat_front_addr": "10.20.0.13:6941\/4213515",
"state": [
"autoout",
"exists"
]
},
{
"osd": 32,
"uuid": "db6f3afa-53ed-453a-97e3-861e88cb818f",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15684,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17873,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6884\/213261",
"cluster_addr": "10.20.0.13:6856\/3213261",
"heartbeat_back_addr": "10.20.0.13:6863\/3213261",
"heartbeat_front_addr": "10.20.0.13:6865\/3213261",
"state": [
"exists",
"up"
]
},
{
"osd": 33,
"uuid": "d5e59852-06b4-4a30-8c5e-ff7e328b5455",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15692,
"last_clean_end": 16125,
"up_from": 16130,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6900\/213450",
"cluster_addr": "10.20.0.13:6936\/3213450",
"heartbeat_back_addr": "10.20.0.13:6938\/3213450",
"heartbeat_front_addr": "10.20.0.13:6939\/3213450",
"state": [
"autoout",
"exists"
]
},
{
"osd": 34,
"uuid": "f35a10c5-217a-4cfb-88b9-7334bda441b8",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15687,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6831\/212981",
"cluster_addr": "10.20.0.13:6942\/2212981",
"heartbeat_back_addr": "10.20.0.13:6943\/2212981",
"heartbeat_front_addr": "10.20.0.13:6944\/2212981",
"state": [
"autoout",
"exists"
]
},
{
"osd": 35,
"uuid": "335e797f-a390-4f08-9da6-9ab76ffb12ae",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15687,
"last_clean_end": 16109,
"up_from": 16112,
"up_thru": 17872,
"down_at": 16109,
"lost_at": 0,
"public_addr": "10.20.0.13:6877\/213258",
"cluster_addr": "10.20.0.13:6803\/2213258",
"heartbeat_back_addr": "10.20.0.13:6832\/2213258",
"heartbeat_front_addr": "10.20.0.13:6833\/2213258",
"state": [
"exists",
"up"
]
},
{
"osd": 36,
"uuid": "33c11fa1-1b03-42e4-8296-dc55ba052b35",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 18112,
"last_clean_end": 18118,
"up_from": 18121,
"up_thru": 18121,
"down_at": 18120,
"lost_at": 0,
"public_addr": "10.20.0.12:6809\/3785178",
"cluster_addr": "10.20.0.12:6810\/3785178",
"heartbeat_back_addr": "10.20.0.12:6811\/3785178",
"heartbeat_front_addr": "10.20.0.12:6812\/3785178",
"state": [
"exists",
"up"
]
}
],
"osd_xinfo": [
{
"osd": 0,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.927101,
"laggy_interval": 46,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 1,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.896128,
"laggy_interval": 43,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 2,
"down_stamp": "2015-04-26 14:39:36.274440",
"laggy_probability": 0.882482,
"laggy_interval": 68,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 3,
"down_stamp": "2015-04-26 14:39:41.282378",
"laggy_probability": 0.943535,
"laggy_interval": 41,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 4,
"down_stamp": "2015-04-26 14:39:46.308944",
"laggy_probability": 0.917737,
"laggy_interval": 56,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 5,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.882482,
"laggy_interval": 15,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 6,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.917737,
"laggy_interval": 59,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 7,
"down_stamp": "2015-04-26 14:39:46.308944",
"laggy_probability": 0.917737,
"laggy_interval": 95,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 8,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.642416,
"laggy_interval": 58,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 9,
"down_stamp": "2015-04-26 14:39:51.653157",
"laggy_probability": 0.919336,
"laggy_interval": 24,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 10,
"down_stamp": "2015-04-26 14:39:56.661476",
"laggy_probability": 0.880998,
"laggy_interval": 69,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 11,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.604091,
"laggy_interval": 32,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 12,
"down_stamp": "2015-04-26 14:39:56.661476",
"laggy_probability": 0.849601,
"laggy_interval": 59,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 13,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.859651,
"laggy_interval": 42,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 14,
"down_stamp": "2015-04-26 14:40:01.673316",
"laggy_probability": 0.948316,
"laggy_interval": 30,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 15,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.945507,
"laggy_interval": 26,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 16,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.948413,
"laggy_interval": 19,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 17,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.773384,
"laggy_interval": 72,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 18,
"down_stamp": "2015-04-26 14:40:06.682693",
"laggy_probability": 0.889015,
"laggy_interval": 50,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 19,
"down_stamp": "2015-04-26 03:22:33.512773",
"laggy_probability": 0.931228,
"laggy_interval": 29,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 20,
"down_stamp": "2015-04-26 03:22:55.024663",
"laggy_probability": 0.894947,
"laggy_interval": 43,
"features": 1125899906842623,
"old_weight": 0
},
{
"osd": 21,
"down_stamp": "2015-04-26 03:22:41.016805",
"laggy_probability": 0.933753,
"laggy_interval": 14,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 22,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.951671,
"laggy_interval": 18,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 23,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.849840,
"laggy_interval": 65,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 24,
"down_stamp": "2015-04-26 02:03:26.390577",
"laggy_probability": 0.806616,
"laggy_interval": 47,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 25,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.810979,
"laggy_interval": 52,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 26,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.858681,
"laggy_interval": 132,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 27,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.796432,
"laggy_interval": 70,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 28,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.908335,
"laggy_interval": 39,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 29,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.732148,
"laggy_interval": 53,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 30,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.866609,
"laggy_interval": 44,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 31,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.866052,
"laggy_interval": 46,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 32,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.812573,
"laggy_interval": 53,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 33,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.797734,
"laggy_interval": 62,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 34,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.681026,
"laggy_interval": 108,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 35,
"down_stamp": "2015-04-26 02:03:26.390577",
"laggy_probability": 0.703038,
"laggy_interval": 47,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 36,
"down_stamp": "2015-04-27 16:35:38.560828",
"laggy_probability": 0.286094,
"laggy_interval": 6,
"features": 1125899906842623,
"old_weight": 0
}
],
"pg_temp": [],
"primary_temp": [],
"blacklist": [
"2015-04-27 17:00:12.409573"
],
"erasure_code_profiles": {
"default": {
"directory": "\/usr\/lib\/ceph\/erasure-code",
"k": "2",
"m": "1",
"plugin": "jerasure",
"technique": "reed_sol_van"
}
}
}
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
