Hi Here's the file. The cluster was working totally fine for months before this, no problems at all. 'ceph osd tree' still shows 11 osd's up, even though all of them are down. If I add a new one, that will get added to the list and it goes up, but when I stop it, the status will still show it as up. Here's a log for that osd http://beta.xaasbox.com/ceph/ceph-osd.37.log Br, Tuomas -----Original Message----- From: Sage Weil [mailto:sweil@xxxxxxxxxx] Sent: 28. huhtikuuta 2015 20:02 To: Tuomas Juntunen Cc: ceph-users@xxxxxxxxxxxxxx Subject: RE: Upgrade from Giant to Hammer and after some basic operations most of the OSD's went down On Tue, 28 Apr 2015, Tuomas Juntunen wrote: > Just to add some more interesting behavior to my problem, is that > monitors are not updating the status of OSD's. Yeah, this is very strange. I see 2015-04-27 22:25:26.142245 7f78d793a900 10 osd.15 17882 _get_pool 4 cached_removed_snaps [1~1,4~a,f~2,12~10,23~4,28~2,2e~2,32~c,3f~4] but pool 4 (images) has no snaps and a zeroed snap_seq in the osdmap dump you provided. Can you attach ceph osd dump -f json-pretty 17882 ? The fact that your mon status isn't changing makes me wonder if these are two different ceph clusters or something similarly silly... if the mds daemons are down the status should show stale. If all osds stop you might not see the osds go down for ~10 minutes (since they aren't monitoring each other). sage > > Even when I stop all the remaining OSD's, ceph osd tree shows them as up. > Also there the status of mons and mds doesn't seem to update correctly > in my opinion. > > Below is a copy of status when one mon and mds are stopped and all of > the osd's are also stopped. > > monmap e7: 3 mons at > {ceph1=10.20.0.11:6789/0,ceph2=10.20.0.12:6789/0,ceph3=10.20.0.13:6789/0} > election epoch 48, quorum 0,1 ceph1,ceph2 > mdsmap e1750: 1/1/1 up {0=ceph2=up:replay}, 1 up:standby > osdmap e18132: 37 osds: 11 up, 11 in > > Br, > Tuomas > > > -----Original Message----- > From: Sage Weil [mailto:sweil@xxxxxxxxxx] > Sent: 27. huhtikuuta 2015 22:22 > To: Tuomas Juntunen > Cc: ceph-users@xxxxxxxxxxxxxx > Subject: RE: Upgrade from Giant to Hammer and after some > basic operations most of the OSD's went down > > On Mon, 27 Apr 2015, Tuomas Juntunen wrote: > > Hey > > > > Got the log, you can get it from > > http://beta.xaasbox.com/ceph/ceph-osd.15.log > > Can you repeat this with 'debug osd = 20'? Thanks! > > sage > > > > > Br, > > Tuomas > > > > > > -----Original Message----- > > From: Sage Weil [mailto:sweil@xxxxxxxxxx] > > Sent: 27. huhtikuuta 2015 20:45 > > To: Tuomas Juntunen > > Cc: ceph-users@xxxxxxxxxxxxxx > > Subject: Re: Upgrade from Giant to Hammer and after > > some basic operations most of the OSD's went down > > > > Yeah, no snaps: > > > > images: > > "snap_mode": "selfmanaged", > > "snap_seq": 0, > > "snap_epoch": 17882, > > "pool_snaps": [], > > "removed_snaps": "[]", > > > > img: > > "snap_mode": "selfmanaged", > > "snap_seq": 0, > > "snap_epoch": 0, > > "pool_snaps": [], > > "removed_snaps": "[]", > > > > ...and actually the log shows this happens on pool 2 (rbd), which > > has > > > > "snap_mode": "selfmanaged", > > "snap_seq": 0, > > "snap_epoch": 0, > > "pool_snaps": [], > > "removed_snaps": "[]", > > > > I'm guessin gthe offending code is > > > > pi->build_removed_snaps(newly_removed_snaps); > > newly_removed_snaps.subtract(cached_removed_snaps); > > > > so newly_removed_snaps should be empty, and apparently > > cached_removed_snaps is not? Maybe one of your older osdmaps has > > snap info for rbd? It doesn't make sense. :/ Maybe > > > > ceph osd dump 18127 -f json-pretty > > > > just to be certain? I've pushed a branch 'wip-hammer-snaps' that > > will appear at gitbuilder.ceph.com in 20-30 minutes that will output > > some additional debug info. It will be at > > > > > > http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/ref/wip-hamm > > er > > -sanps > > > > or similar, depending on your distro. Can you install it one on > > node and start and osd with logging to reproduce the crash? > > > > Thanks! > > sage > > > > > > On Mon, 27 Apr 2015, Tuomas Juntunen wrote: > > > > > Hi > > > > > > Here you go > > > > > > Br, > > > Tuomas > > > > > > > > > > > > -----Original Message----- > > > From: Sage Weil [mailto:sweil@xxxxxxxxxx] > > > Sent: 27. huhtikuuta 2015 19:23 > > > To: Tuomas Juntunen > > > Cc: 'Samuel Just'; ceph-users@xxxxxxxxxxxxxx > > > Subject: Re: Upgrade from Giant to Hammer and after > > > some basic operations most of the OSD's went down > > > > > > On Mon, 27 Apr 2015, Tuomas Juntunen wrote: > > > > Thanks for the info. > > > > > > > > For my knowledge there was no snapshots on that pool, but cannot > > > > verify that. > > > > > > Can you attach a 'ceph osd dump -f json-pretty'? That will shed a > > > bit more light on what happened (and the simplest way to fix it). > > > > > > sage > > > > > > > > > > Any way to make this work again? Removing the tier and other > > > > settings didn't fix it, I tried it the second this happened. > > > > > > > > Br, > > > > Tuomas > > > > > > > > -----Original Message----- > > > > From: Samuel Just [mailto:sjust@xxxxxxxxxx] > > > > Sent: 27. huhtikuuta 2015 15:50 > > > > To: tuomas juntunen > > > > Cc: ceph-users@xxxxxxxxxxxxxx > > > > Subject: Re: Upgrade from Giant to Hammer and after > > > > some basic operations most of the OSD's went down > > > > > > > > So, the base tier is what determines the snapshots for the > > > > cache/base pool > > > amalgam. You added a populated pool complete with snapshots on > > > top of a base tier without snapshots. Apparently, it caused an > > > existential crisis for the snapshot code. That's one of the > > > reasons why there is a --force-nonempty flag for that operation, I > > > think. I think the immediate answer is probably to disallow pools > > > with snapshots as a cache tier altogether until we think of a good > > > way to > make it work. > > > > -Sam > > > > > > > > ----- Original Message ----- > > > > From: "tuomas juntunen" <tuomas.juntunen@xxxxxxxxxxxxxxx> > > > > To: "Samuel Just" <sjust@xxxxxxxxxx> > > > > Cc: ceph-users@xxxxxxxxxxxxxx > > > > Sent: Monday, April 27, 2015 4:56:58 AM > > > > Subject: Re: Upgrade from Giant to Hammer and after > > > > some basic operations most of the OSD's went down > > > > > > > > > > > > > > > > The following: > > > > > > > > ceph osd tier add img images --force-nonempty ceph osd tier > > > > cache-mode images forward ceph osd tier set-overlay img images > > > > > > > > Idea was to make images as a tier to img, move data to img then > > > > change > > > clients to use the new img pool. > > > > > > > > Br, > > > > Tuomas > > > > > > > > > Can you explain exactly what you mean by: > > > > > > > > > > "Also I created one pool for tier to be able to move data > > > > > without > > > outage." > > > > > > > > > > -Sam > > > > > ----- Original Message ----- > > > > > From: "tuomas juntunen" <tuomas.juntunen@xxxxxxxxxxxxxxx> > > > > > To: "Ian Colle" <icolle@xxxxxxxxxx> > > > > > Cc: ceph-users@xxxxxxxxxxxxxx > > > > > Sent: Monday, April 27, 2015 4:23:44 AM > > > > > Subject: Re: Upgrade from Giant to Hammer and > > > > > after some basic operations most of the OSD's went down > > > > > > > > > > Hi > > > > > > > > > > Any solution for this yet? > > > > > > > > > > Br, > > > > > Tuomas > > > > > > > > > >> It looks like you may have hit > > > > >> http://tracker.ceph.com/issues/7915 > > > > >> > > > > >> Ian R. Colle > > > > >> Global Director > > > > >> of Software Engineering > > > > >> Red Hat (Inktank is now part of Red Hat!) > > > > >> http://www.linkedin.com/in/ircolle > > > > >> http://www.twitter.com/ircolle > > > > >> Cell: +1.303.601.7713 > > > > >> Email: icolle@xxxxxxxxxx > > > > >> > > > > >> ----- Original Message ----- > > > > >> From: "tuomas juntunen" <tuomas.juntunen@xxxxxxxxxxxxxxx> > > > > >> To: ceph-users@xxxxxxxxxxxxxx > > > > >> Sent: Monday, April 27, 2015 1:56:29 PM > > > > >> Subject: Upgrade from Giant to Hammer and after > > > > >> some basic operations most of the OSD's went down > > > > >> > > > > >> > > > > >> > > > > >> I upgraded Ceph from 0.87 Giant to 0.94.1 Hammer > > > > >> > > > > >> Then created new pools and deleted some old ones. Also I > > > > >> created one pool for tier to be able to move data without outage. > > > > >> > > > > >> After these operations all but 10 OSD's are down and creating > > > > >> this kind of messages to logs, I get more than 100gb of these > > > > >> in a > > night: > > > > >> > > > > >> -19> 2015-04-27 10:17:08.808584 7fd8e748d700 5 osd.23 pg_epoch: > > > > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 > > > > >> ec=1 les/c > > > > >> 16609/16659 > > > > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42 > > > > >> crt=8480'7 lcod > > > > >> 0'0 inactive NOTIFY] enter Started > > > > >> -18> 2015-04-27 10:17:08.808596 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 > > > > >> ec=1 les/c > > > > >> 16609/16659 > > > > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42 > > > > >> crt=8480'7 lcod > > > > >> 0'0 inactive NOTIFY] enter Start > > > > >> -17> 2015-04-27 10:17:08.808608 7fd8e748d700 1 osd.23 pg_epoch: > > > > > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 > > > > >> ec=1 les/c > > > > >> 16609/16659 > > > > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42 > > > > >> crt=8480'7 lcod > > > > >> 0'0 inactive NOTIFY] state<Start>: transitioning to Stray > > > > >> -16> 2015-04-27 10:17:08.808621 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 > > > > >> ec=1 les/c > > > > >> 16609/16659 > > > > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42 > > > > >> crt=8480'7 lcod > > > > >> 0'0 inactive NOTIFY] exit Start 0.000025 0 0.000000 > > > > >> -15> 2015-04-27 10:17:08.808637 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 > > > > >> ec=1 les/c > > > > >> 16609/16659 > > > > >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42 > > > > >> crt=8480'7 lcod > > > > >> 0'0 inactive NOTIFY] enter Started/Stray > > > > >> -14> 2015-04-27 10:17:08.808796 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c > > > > >> 17879/17879 > > > > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive > > > > >> NOTIFY] exit Reset 0.119467 4 0.000037 > > > > >> -13> 2015-04-27 10:17:08.808817 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c > > > > >> 17879/17879 > > > > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive > > > > >> NOTIFY] enter Started > > > > >> -12> 2015-04-27 10:17:08.808828 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c > > > > >> 17879/17879 > > > > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive > > > > >> NOTIFY] enter Start > > > > >> -11> 2015-04-27 10:17:08.808838 7fd8e748d700 1 osd.23 pg_epoch: > > > > > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c > > > > >> 17879/17879 > > > > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive > > > > >> NOTIFY] > > > > >> state<Start>: transitioning to Stray > > > > >> -10> 2015-04-27 10:17:08.808849 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c > > > > >> 17879/17879 > > > > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive > > > > >> NOTIFY] exit Start 0.000020 0 0.000000 > > > > >> -9> 2015-04-27 10:17:08.808861 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c > > > > >> 17879/17879 > > > > >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive > > > > >> NOTIFY] enter Started/Stray > > > > >> -8> 2015-04-27 10:17:08.809427 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> inactive] exit Reset 7.511623 45 0.000165 > > > > >> -7> 2015-04-27 10:17:08.809445 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> inactive] enter Started > > > > >> -6> 2015-04-27 10:17:08.809456 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> inactive] enter Start > > > > >> -5> 2015-04-27 10:17:08.809468 7fd8e748d700 1 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> inactive] > > > > >> state<Start>: transitioning to Primary > > > > >> -4> 2015-04-27 10:17:08.809479 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> inactive] exit Start 0.000023 0 0.000000 > > > > >> -3> 2015-04-27 10:17:08.809492 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> inactive] enter Started/Primary > > > > >> -2> 2015-04-27 10:17:08.809502 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> inactive] enter Started/Primary/Peering > > > > >> -1> 2015-04-27 10:17:08.809513 7fd8e748d700 5 osd.23 pg_epoch: > > > > > >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c > > > > >> 16127/16344 > > > > >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 > > > > >> peering] enter Started/Primary/Peering/GetInfo > > > > >> 0> 2015-04-27 10:17:08.813837 7fd8e748d700 -1 > > > ./include/interval_set.h: > > > > >> In > > > > >> function 'void interval_set<T>::erase(T, T) [with T = snapid_t]' > > > > >> thread > > > > >> 7fd8e748d700 time 2015-04-27 10:17:08.809899 > > > > >> ./include/interval_set.h: 385: FAILED assert(_size >= 0) > > > > >> > > > > >> ceph version 0.94.1 > > > > >> (e4bfad3a3c51054df7e537a724c8d0bf9be972ff) > > > > >> 1: (ceph::__ceph_assert_fail(char const*, char const*, int, > > > > >> char > > > > >> const*)+0x8b) > > > > >> [0xbc271b] > > > > >> 2: (interval_set<snapid_t>::subtract(interval_set<snapid_t> > > > > >> const&)+0xb0) [0x82cd50] > > > > >> 3: (PGPool::update(std::tr1::shared_ptr<OSDMap > > > > >> const>)+0x52e) [0x80113e] > > > > >> 4: (PG::handle_advance_map(std::tr1::shared_ptr<OSDMap > > > > >> const>, std::tr1::shared_ptr<OSDMap const>, std::vector<int, > > > > >> std::allocator<int> >&, int, std::vector<int, > > > > >> std::allocator<int> > > > > >> >&, int, PG::RecoveryCtx*)+0x282) [0x801652] > > > > >> 5: (OSD::advance_pg(unsigned int, PG*, > > > > >> ThreadPool::TPHandle&, PG::RecoveryCtx*, > > > > >> std::set<boost::intrusive_ptr<PG>, > > > > >> std::less<boost::intrusive_ptr<PG> >, > > > > >> std::allocator<boost::intrusive_ptr<PG> > >*)+0x2c3) > > > > >> [0x6b0e43] > > > > >> 6: (OSD::process_peering_events(std::list<PG*, > > > > >> std::allocator<PG*> > > > > >> > const&, > > > > >> ThreadPool::TPHandle&)+0x21c) [0x6b191c] > > > > >> 7: (OSD::PeeringWQ::_process(std::list<PG*, > > > > >> std::allocator<PG*> > > > > >> > const&, > > > > >> ThreadPool::TPHandle&)+0x18) [0x709278] > > > > >> 8: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa5e) > > > > >> [0xbb38ae] > > > > >> 9: (ThreadPool::WorkThread::entry()+0x10) [0xbb4950] > > > > >> 10: (()+0x8182) [0x7fd906946182] > > > > >> 11: (clone()+0x6d) [0x7fd904eb147d] > > > > >> > > > > >> Also by monitoring (ceph -w) I get the following messages, > > > > >> also lots of > > > them. > > > > >> > > > > >> 2015-04-27 10:39:52.935812 mon.0 [INF] from='client.? > > > 10.20.0.13:0/1174409' > > > > >> entity='osd.30' cmd=[{"prefix": "osd crush create-or-move", "args": > > > > >> ["host=ceph3", "root=default"], "id": 30, "weight": 1.82}]: > > > > >> dispatch > > > > >> 2015-04-27 10:39:53.297376 mon.0 [INF] from='client.? > > > 10.20.0.13:0/1174483' > > > > >> entity='osd.26' cmd=[{"prefix": "osd crush create-or-move", "args": > > > > >> ["host=ceph3", "root=default"], "id": 26, "weight": 1.82}]: > > > > >> dispatch > > > > >> > > > > >> > > > > >> This is a cluster of 3 nodes with 36 OSD's, nodes are also > > > > >> mons and mds's to save servers. All run Ubuntu 14.04.2. > > > > >> > > > > >> I have pretty much tried everything I could think of. > > > > >> > > > > >> Restarting daemons doesn't help. > > > > >> > > > > >> Any help would be appreciated. I can also provide more logs > > > > >> if necessary. They just seem to get pretty large in few moments. > > > > >> > > > > >> Thank you > > > > >> Tuomas > > > > >> > > > > >> > > > > >> _______________________________________________ > > > > >> ceph-users mailing list > > > > >> ceph-users@xxxxxxxxxxxxxx > > > > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > > > _______________________________________________ > > > > > ceph-users mailing list > > > > > ceph-users@xxxxxxxxxxxxxx > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > ceph-users mailing list > > > > ceph-users@xxxxxxxxxxxxxx > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > > > > > > > _______________________________________________ > > > > ceph-users mailing list > > > > ceph-users@xxxxxxxxxxxxxx > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > > > > > > > >
{ "epoch": 17882, "fsid": "a2974742-3805-4cd3-bc79-765f2bddaefe", "created": "2014-10-15 20:43:45.186949", "modified": "2015-04-26 14:19:23.857402", "flags": "", "cluster_snapshot": "", "pool_max": 13, "max_osd": 71, "pools": [ { "pool": 0, "pool_name": "data", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 1, "crush_ruleset": 0, "object_hash": 2, "pg_num": 4096, "pg_placement_num": 4096, "crash_replay_interval": 45, "last_change": "1112", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 0, "cache_target_full_ratio_micro": 0, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 1, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 1, "pool_name": "metadata", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 1, "crush_ruleset": 0, "object_hash": 2, "pg_num": 4096, "pg_placement_num": 4096, "crash_replay_interval": 0, "last_change": "1114", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 0, "cache_target_full_ratio_micro": 0, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 1, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 2, "pool_name": "rbd", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 2, "min_size": 1, "crush_ruleset": 0, "object_hash": 2, "pg_num": 4096, "pg_placement_num": 4096, "crash_replay_interval": 0, "last_change": "1116", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 0, "cache_target_full_ratio_micro": 0, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 1, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 3, "pool_name": "volumes", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 1, "crush_ruleset": 0, "object_hash": 2, "pg_num": 4096, "pg_placement_num": 4096, "crash_replay_interval": 0, "last_change": "9974", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 23, "snap_epoch": 9974, "pool_snaps": [], "removed_snaps": "[1~17]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "default", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 1, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 4, "pool_name": "images", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 1, "crush_ruleset": 0, "object_hash": 2, "pg_num": 4096, "pg_placement_num": 4096, "crash_replay_interval": 0, "last_change": "17882", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 17882, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": 9, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "default", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 1, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 6, "pool_name": "vms", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 1, "crush_ruleset": 0, "object_hash": 2, "pg_num": 4096, "pg_placement_num": 4096, "crash_replay_interval": 0, "last_change": "1122", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "default", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 1, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 7, "pool_name": "san", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 1, "crush_ruleset": 0, "object_hash": 2, "pg_num": 4096, "pg_placement_num": 4096, "crash_replay_interval": 0, "last_change": "14096", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 0, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 8, "pool_name": "vol-ssd-accelerated", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 2, "crush_ruleset": 0, "object_hash": 2, "pg_num": 1024, "pg_placement_num": 1024, "crash_replay_interval": 0, "last_change": "17861", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 0, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 9, "pool_name": "img", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 2, "crush_ruleset": 0, "object_hash": 2, "pg_num": 256, "pg_placement_num": 256, "crash_replay_interval": 0, "last_change": "17882", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 17882, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [ 4 ], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 0, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 10, "pool_name": "vm", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 2, "crush_ruleset": 0, "object_hash": 2, "pg_num": 1024, "pg_placement_num": 1024, "crash_replay_interval": 0, "last_change": "17863", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 0, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 11, "pool_name": "infradisks", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 2, "crush_ruleset": 0, "object_hash": 2, "pg_num": 256, "pg_placement_num": 256, "crash_replay_interval": 0, "last_change": "17865", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 0, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 12, "pool_name": "data-new", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 2, "crush_ruleset": 0, "object_hash": 2, "pg_num": 256, "pg_placement_num": 256, "crash_replay_interval": 0, "last_change": "17869", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 0, "stripe_width": 0, "expected_num_objects": 0 }, { "pool": 13, "pool_name": "metadata-new", "flags": 1, "flags_names": "hashpspool", "type": 1, "size": 3, "min_size": 2, "crush_ruleset": 0, "object_hash": 2, "pg_num": 256, "pg_placement_num": 256, "crash_replay_interval": 0, "last_change": "17870", "last_force_op_resend": "0", "auid": 0, "snap_mode": "selfmanaged", "snap_seq": 0, "snap_epoch": 0, "pool_snaps": [], "removed_snaps": "[]", "quota_max_bytes": 0, "quota_max_objects": 0, "tiers": [], "tier_of": -1, "read_tier": -1, "write_tier": -1, "cache_mode": "none", "target_max_bytes": 0, "target_max_objects": 0, "cache_target_dirty_ratio_micro": 400000, "cache_target_full_ratio_micro": 800000, "cache_min_flush_age": 0, "cache_min_evict_age": 0, "erasure_code_profile": "", "hit_set_params": { "type": "none" }, "hit_set_period": 0, "hit_set_count": 0, "min_read_recency_for_promote": 0, "stripe_width": 0, "expected_num_objects": 0 } ], "osds": [ { "osd": 0, "uuid": "757c3bc5-4d00-4344-8de4-82f5379c96af", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15738, "last_clean_end": 16079, "up_from": 16081, "up_thru": 17872, "down_at": 16079, "lost_at": 0, "public_addr": "10.20.0.11:6981\/245797", "cluster_addr": "10.20.0.11:6881\/5245797", "heartbeat_back_addr": "10.20.0.11:6883\/5245797", "heartbeat_front_addr": "10.20.0.11:6887\/5245797", "state": [ "exists", "up" ] }, { "osd": 1, "uuid": "c7eaa4ac-99fc-46db-84aa-a67274896ec8", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15740, "last_clean_end": 16079, "up_from": 16083, "up_thru": 17873, "down_at": 16079, "lost_at": 0, "public_addr": "10.20.0.11:6813\/245273", "cluster_addr": "10.20.0.11:6888\/4245273", "heartbeat_back_addr": "10.20.0.11:6890\/4245273", "heartbeat_front_addr": "10.20.0.11:6894\/4245273", "state": [ "exists", "up" ] }, { "osd": 2, "uuid": "206b2949-4adf-4789-8e06-f68a8ee819c9", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15739, "last_clean_end": 16079, "up_from": 16081, "up_thru": 17874, "down_at": 16079, "lost_at": 0, "public_addr": "10.20.0.11:6808\/245269", "cluster_addr": "10.20.0.11:6811\/4245269", "heartbeat_back_addr": "10.20.0.11:6823\/4245269", "heartbeat_front_addr": "10.20.0.11:6831\/4245269", "state": [ "exists", "up" ] }, { "osd": 3, "uuid": "90b7c219-4dcd-48ea-a24d-f3b796a521e4", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15736, "last_clean_end": 16125, "up_from": 16128, "up_thru": 17873, "down_at": 16125, "lost_at": 0, "public_addr": "10.20.0.11:6848\/245357", "cluster_addr": "10.20.0.11:6886\/6245357", "heartbeat_back_addr": "10.20.0.11:6891\/6245357", "heartbeat_front_addr": "10.20.0.11:6892\/6245357", "state": [ "exists", "up" ] }, { "osd": 4, "uuid": "049ef94f-121a-4e71-8ba6-27eaebf0a569", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15737, "last_clean_end": 16123, "up_from": 16125, "up_thru": 17871, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.11:6854\/245375", "cluster_addr": "10.20.0.11:6849\/5245375", "heartbeat_back_addr": "10.20.0.11:6850\/5245375", "heartbeat_front_addr": "10.20.0.11:6853\/5245375", "state": [ "exists", "up" ] }, { "osd": 5, "uuid": "2437a53b-339e-45af-b4de-0fc675d27405", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15734, "last_clean_end": 16087, "up_from": 16089, "up_thru": 17871, "down_at": 16087, "lost_at": 0, "public_addr": "10.20.0.11:6834\/245323", "cluster_addr": "10.20.0.11:6803\/4245323", "heartbeat_back_addr": "10.20.0.11:6804\/4245323", "heartbeat_front_addr": "10.20.0.11:6805\/4245323", "state": [ "exists", "up" ] }, { "osd": 6, "uuid": "f117ceed-b1fd-4069-99fe-b7aba9f3ef8d", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15738, "last_clean_end": 16079, "up_from": 16083, "up_thru": 17871, "down_at": 16079, "lost_at": 0, "public_addr": "10.20.0.11:6859\/245401", "cluster_addr": "10.20.0.11:6865\/5245401", "heartbeat_back_addr": "10.20.0.11:6868\/5245401", "heartbeat_front_addr": "10.20.0.11:6874\/5245401", "state": [ "exists", "up" ] }, { "osd": 7, "uuid": "e98e9b8a-9c62-4e3e-bdb4-c2c30103c0c1", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15730, "last_clean_end": 16123, "up_from": 16125, "up_thru": 17872, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.11:6801\/245246", "cluster_addr": "10.20.0.11:6812\/5245246", "heartbeat_back_addr": "10.20.0.11:6814\/5245246", "heartbeat_front_addr": "10.20.0.11:6821\/5245246", "state": [ "exists", "up" ] }, { "osd": 8, "uuid": "41e471cd-fafe-4422-8bf5-22018bbe1375", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15736, "last_clean_end": 17783, "up_from": 17795, "up_thru": 17871, "down_at": 17784, "lost_at": 0, "public_addr": "10.20.0.11:6810\/458698", "cluster_addr": "10.20.0.11:6819\/458698", "heartbeat_back_addr": "10.20.0.11:6826\/458698", "heartbeat_front_addr": "10.20.0.11:6833\/458698", "state": [ "exists", "up" ] }, { "osd": 9, "uuid": "d68eeebd-d058-4b1c-a30a-994bf8fc8030", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15733, "last_clean_end": 16079, "up_from": 16081, "up_thru": 17873, "down_at": 16079, "lost_at": 0, "public_addr": "10.20.0.11:6969\/245600", "cluster_addr": "10.20.0.11:6847\/5245600", "heartbeat_back_addr": "10.20.0.11:6860\/5245600", "heartbeat_front_addr": "10.20.0.11:6864\/5245600", "state": [ "exists", "up" ] }, { "osd": 10, "uuid": "660747d6-3f47-449a-bc69-5399b0d54ff6", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15736, "last_clean_end": 16124, "up_from": 16128, "up_thru": 17875, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.11:6822\/245281", "cluster_addr": "10.20.0.11:6855\/4245281", "heartbeat_back_addr": "10.20.0.11:6856\/4245281", "heartbeat_front_addr": "10.20.0.11:6866\/4245281", "state": [ "exists", "up" ] }, { "osd": 11, "uuid": "805965b1-127f-44a6-9a05-8a643eb7a512", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15737, "last_clean_end": 17786, "up_from": 17801, "up_thru": 17870, "down_at": 17787, "lost_at": 0, "public_addr": "10.20.0.11:6835\/458813", "cluster_addr": "10.20.0.11:6836\/458813", "heartbeat_back_addr": "10.20.0.11:6837\/458813", "heartbeat_front_addr": "10.20.0.11:6838\/458813", "state": [ "exists", "up" ] }, { "osd": 12, "uuid": "61fbfcbe-d642-478f-9620-f9d72ee96238", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16126, "up_from": 16129, "up_thru": 17872, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.12:6873\/7542", "cluster_addr": "10.20.0.12:6947\/4007542", "heartbeat_back_addr": "10.20.0.12:6948\/4007542", "heartbeat_front_addr": "10.20.0.12:6949\/4007542", "state": [ "exists", "up" ] }, { "osd": 13, "uuid": "6faad33b-00be-42a4-92ba-08be5ab7f995", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16705, "up_from": 16707, "up_thru": 17871, "down_at": 16703, "lost_at": 0, "public_addr": "10.20.0.12:6821\/7274", "cluster_addr": "10.20.0.12:6859\/5007274", "heartbeat_back_addr": "10.20.0.12:6860\/5007274", "heartbeat_front_addr": "10.20.0.12:6861\/5007274", "state": [ "exists", "up" ] }, { "osd": 14, "uuid": "f301705b-e725-443d-96e1-d9ec9aafe657", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16962, "up_from": 16966, "up_thru": 17871, "down_at": 16962, "lost_at": 0, "public_addr": "10.20.0.12:6809\/7209", "cluster_addr": "10.20.0.12:6803\/7007209", "heartbeat_back_addr": "10.20.0.12:6804\/7007209", "heartbeat_front_addr": "10.20.0.12:6810\/7007209", "state": [ "exists", "up" ] }, { "osd": 15, "uuid": "536bf483-10de-44b0-8e1e-4f349fbe572a", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15785, "last_clean_end": 16700, "up_from": 16715, "up_thru": 17871, "down_at": 16693, "lost_at": 0, "public_addr": "10.20.0.12:6881\/7613", "cluster_addr": "10.20.0.12:6845\/7007613", "heartbeat_back_addr": "10.20.0.12:6849\/7007613", "heartbeat_front_addr": "10.20.0.12:6850\/7007613", "state": [ "exists", "up" ] }, { "osd": 16, "uuid": "4185bd20-8eb0-4616-b36e-bacb181ae40e", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16688, "up_from": 16693, "up_thru": 17871, "down_at": 16686, "lost_at": 0, "public_addr": "10.20.0.12:6813\/7210", "cluster_addr": "10.20.0.12:6814\/7007210", "heartbeat_back_addr": "10.20.0.12:6815\/7007210", "heartbeat_front_addr": "10.20.0.12:6816\/7007210", "state": [ "exists", "up" ] }, { "osd": 17, "uuid": "a6f2f5b4-477f-48f9-9acf-d5b7a6c88b98", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16123, "up_from": 16125, "up_thru": 17872, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.12:6817\/7215", "cluster_addr": "10.20.0.12:6840\/3007215", "heartbeat_back_addr": "10.20.0.12:6843\/3007215", "heartbeat_front_addr": "10.20.0.12:6844\/3007215", "state": [ "exists", "up" ] }, { "osd": 18, "uuid": "b31b0bd8-938a-496d-91bc-19bf4f794f82", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16700, "up_from": 16712, "up_thru": 17872, "down_at": 16700, "lost_at": 0, "public_addr": "10.20.0.12:6805\/7185", "cluster_addr": "10.20.0.12:6829\/5007185", "heartbeat_back_addr": "10.20.0.12:6830\/5007185", "heartbeat_front_addr": "10.20.0.12:6841\/5007185", "state": [ "exists", "up" ] }, { "osd": 19, "uuid": "d76b6bd5-1ef3-436c-a75d-3587c515eb56", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16693, "up_from": 16706, "up_thru": 17872, "down_at": 16693, "lost_at": 0, "public_addr": "10.20.0.12:6885\/7622", "cluster_addr": "10.20.0.12:6802\/7007622", "heartbeat_back_addr": "10.20.0.12:6818\/7007622", "heartbeat_front_addr": "10.20.0.12:6819\/7007622", "state": [ "exists", "up" ] }, { "osd": 20, "uuid": "8e4dd982-a4c5-4ca5-9fc5-243f55c4db57", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16703, "up_from": 16705, "up_thru": 17872, "down_at": 16703, "lost_at": 0, "public_addr": "10.20.0.12:6825\/7316", "cluster_addr": "10.20.0.12:6827\/5007316", "heartbeat_back_addr": "10.20.0.12:6831\/5007316", "heartbeat_front_addr": "10.20.0.12:6851\/5007316", "state": [ "exists", "up" ] }, { "osd": 21, "uuid": "760aaf28-0a34-4bbc-af0c-2654b0a43fff", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16699, "up_from": 16701, "up_thru": 17870, "down_at": 16698, "lost_at": 0, "public_addr": "10.20.0.12:6801\/7180", "cluster_addr": "10.20.0.12:6820\/7007180", "heartbeat_back_addr": "10.20.0.12:6822\/7007180", "heartbeat_front_addr": "10.20.0.12:6828\/7007180", "state": [ "exists", "up" ] }, { "osd": 22, "uuid": "40322a34-ab31-4760-b71e-a7672f812cb3", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16966, "up_from": 16969, "up_thru": 17871, "down_at": 16964, "lost_at": 0, "public_addr": "10.20.0.12:6869\/7509", "cluster_addr": "10.20.0.12:6824\/8007509", "heartbeat_back_addr": "10.20.0.12:6832\/8007509", "heartbeat_front_addr": "10.20.0.12:6834\/8007509", "state": [ "exists", "up" ] }, { "osd": 23, "uuid": "e1d81949-f4b5-4cf2-b6af-dccaaeb30ed7", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16123, "up_from": 16125, "up_thru": 17875, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.12:6877\/7592", "cluster_addr": "10.20.0.12:6826\/4007592", "heartbeat_back_addr": "10.20.0.12:6835\/4007592", "heartbeat_front_addr": "10.20.0.12:6839\/4007592", "state": [ "exists", "up" ] }, { "osd": 24, "uuid": "ede77283-a423-4c6b-9c6e-b0e807c63cb5", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15688, "last_clean_end": 16109, "up_from": 16112, "up_thru": 17871, "down_at": 16109, "lost_at": 0, "public_addr": "10.20.0.13:6921\/213483", "cluster_addr": "10.20.0.13:6815\/3213483", "heartbeat_back_addr": "10.20.0.13:6841\/3213483", "heartbeat_front_addr": "10.20.0.13:6846\/3213483", "state": [ "exists", "up" ] }, { "osd": 25, "uuid": "7cfe85f8-3ae9-493d-9801-025ff6c6265d", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15686, "last_clean_end": 16125, "up_from": 16127, "up_thru": 17871, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6801\/212863", "cluster_addr": "10.20.0.13:6918\/3212863", "heartbeat_back_addr": "10.20.0.13:6932\/3212863", "heartbeat_front_addr": "10.20.0.13:6934\/3212863", "state": [ "exists", "up" ] }, { "osd": 26, "uuid": "266f6d70-519f-4c24-bca2-236495a600a7", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15692, "last_clean_end": 16125, "up_from": 16130, "up_thru": 17872, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6917\/213469", "cluster_addr": "10.20.0.13:6896\/4213469", "heartbeat_back_addr": "10.20.0.13:6945\/4213469", "heartbeat_front_addr": "10.20.0.13:6946\/4213469", "state": [ "exists", "up" ] }, { "osd": 27, "uuid": "68644fa9-9459-4db0-a6c9-01661645038b", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15689, "last_clean_end": 16125, "up_from": 16127, "up_thru": 17871, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6899\/213448", "cluster_addr": "10.20.0.13:6876\/3213448", "heartbeat_back_addr": "10.20.0.13:6902\/3213448", "heartbeat_front_addr": "10.20.0.13:6904\/3213448", "state": [ "exists", "up" ] }, { "osd": 28, "uuid": "fc3d5749-7673-4100-a0d4-f25e9cc0bc88", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15688, "last_clean_end": 16125, "up_from": 16127, "up_thru": 17871, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6892\/213419", "cluster_addr": "10.20.0.13:6866\/5213419", "heartbeat_back_addr": "10.20.0.13:6867\/5213419", "heartbeat_front_addr": "10.20.0.13:6873\/5213419", "state": [ "exists", "up" ] }, { "osd": 29, "uuid": "cb5feda9-de3f-4e42-bb73-7945b4928b22", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15689, "last_clean_end": 16109, "up_from": 16113, "up_thru": 17872, "down_at": 16109, "lost_at": 0, "public_addr": "10.20.0.13:6937\/213527", "cluster_addr": "10.20.0.13:6898\/2213527", "heartbeat_back_addr": "10.20.0.13:6926\/2213527", "heartbeat_front_addr": "10.20.0.13:6927\/2213527", "state": [ "exists", "up" ] }, { "osd": 30, "uuid": "ef1e65bb-a634-4096-9466-1262af55db01", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15693, "last_clean_end": 16125, "up_from": 16127, "up_thru": 17872, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6910\/213454", "cluster_addr": "10.20.0.13:6823\/4213454", "heartbeat_back_addr": "10.20.0.13:6838\/4213454", "heartbeat_front_addr": "10.20.0.13:6852\/4213454", "state": [ "exists", "up" ] }, { "osd": 31, "uuid": "3dad6393-67a8-43d4-ba8d-ffd320827396", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15688, "last_clean_end": 16125, "up_from": 16127, "up_thru": 17871, "down_at": 16125, "lost_at": 0, "public_addr": "10.20.0.13:6929\/213515", "cluster_addr": "10.20.0.13:6909\/4213515", "heartbeat_back_addr": "10.20.0.13:6940\/4213515", "heartbeat_front_addr": "10.20.0.13:6941\/4213515", "state": [ "exists", "up" ] }, { "osd": 32, "uuid": "db6f3afa-53ed-453a-97e3-861e88cb818f", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15684, "last_clean_end": 16125, "up_from": 16127, "up_thru": 17873, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6884\/213261", "cluster_addr": "10.20.0.13:6856\/3213261", "heartbeat_back_addr": "10.20.0.13:6863\/3213261", "heartbeat_front_addr": "10.20.0.13:6865\/3213261", "state": [ "exists", "up" ] }, { "osd": 33, "uuid": "d5e59852-06b4-4a30-8c5e-ff7e328b5455", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15692, "last_clean_end": 16125, "up_from": 16130, "up_thru": 17872, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6900\/213450", "cluster_addr": "10.20.0.13:6936\/3213450", "heartbeat_back_addr": "10.20.0.13:6938\/3213450", "heartbeat_front_addr": "10.20.0.13:6939\/3213450", "state": [ "exists", "up" ] }, { "osd": 34, "uuid": "f35a10c5-217a-4cfb-88b9-7334bda441b8", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15687, "last_clean_end": 16125, "up_from": 16127, "up_thru": 17871, "down_at": 16123, "lost_at": 0, "public_addr": "10.20.0.13:6831\/212981", "cluster_addr": "10.20.0.13:6942\/2212981", "heartbeat_back_addr": "10.20.0.13:6943\/2212981", "heartbeat_front_addr": "10.20.0.13:6944\/2212981", "state": [ "exists", "up" ] }, { "osd": 35, "uuid": "335e797f-a390-4f08-9da6-9ab76ffb12ae", "up": 1, "in": 1, "weight": 1.000000, "primary_affinity": 1.000000, "last_clean_begin": 15687, "last_clean_end": 16109, "up_from": 16112, "up_thru": 17872, "down_at": 16109, "lost_at": 0, "public_addr": "10.20.0.13:6877\/213258", "cluster_addr": "10.20.0.13:6803\/2213258", "heartbeat_back_addr": "10.20.0.13:6832\/2213258", "heartbeat_front_addr": "10.20.0.13:6833\/2213258", "state": [ "exists", "up" ] }, { "osd": 36, "uuid": "10be14b2-1bc9-4aaf-8d5c-1f7aed2eadc5", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15727, "last_clean_end": 16126, "up_from": 17436, "up_thru": 17652, "down_at": 17677, "lost_at": 0, "public_addr": "10.20.0.11:6818\/448672", "cluster_addr": "10.20.0.11:6819\/448672", "heartbeat_back_addr": "10.20.0.11:6820\/448672", "heartbeat_front_addr": "10.20.0.11:6824\/448672", "state": [ "exists" ] }, { "osd": 37, "uuid": "266fe6ea-0152-4584-a8e0-3fb5d0c2f04c", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15721, "last_clean_end": 16127, "up_from": 16512, "up_thru": 16968, "down_at": 17678, "lost_at": 0, "public_addr": "10.20.0.11:6809\/440793", "cluster_addr": "10.20.0.11:6810\/440793", "heartbeat_back_addr": "10.20.0.11:6816\/440793", "heartbeat_front_addr": "10.20.0.11:6817\/440793", "state": [ "exists" ] }, { "osd": 38, "uuid": "9011180a-2de1-4c3a-9f7c-e917878be18b", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15724, "last_clean_end": 16127, "up_from": 17663, "up_thru": 17671, "down_at": 17679, "lost_at": 0, "public_addr": "10.20.0.11:6825\/451446", "cluster_addr": "10.20.0.11:6826\/451446", "heartbeat_back_addr": "10.20.0.11:6827\/451446", "heartbeat_front_addr": "10.20.0.11:6828\/451446", "state": [ "exists" ] }, { "osd": 39, "uuid": "fe3819a6-ed64-4a4f-9b76-791f43d32b10", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 17689, "last_clean_end": 17710, "up_from": 17821, "up_thru": 17700, "down_at": 17835, "lost_at": 0, "public_addr": "10.20.0.11:6820\/460064", "cluster_addr": "10.20.0.11:6824\/460064", "heartbeat_back_addr": "10.20.0.11:6825\/460064", "heartbeat_front_addr": "10.20.0.11:6827\/460064", "state": [ "exists" ] }, { "osd": 40, "uuid": "5f8a9ce2-3efb-4d9c-8c2f-e45ebe6e4990", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15730, "last_clean_end": 16130, "up_from": 17687, "up_thru": 16130, "down_at": 17712, "lost_at": 0, "public_addr": "10.20.0.11:6818\/452335", "cluster_addr": "10.20.0.11:6819\/452335", "heartbeat_back_addr": "10.20.0.11:6820\/452335", "heartbeat_front_addr": "10.20.0.11:6824\/452335", "state": [ "exists" ] }, { "osd": 41, "uuid": "79791990-10e4-4437-ba86-bdf1165700c8", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15727, "last_clean_end": 16131, "up_from": 17689, "up_thru": 17701, "down_at": 17713, "lost_at": 0, "public_addr": "10.20.0.11:6825\/452386", "cluster_addr": "10.20.0.11:6826\/452386", "heartbeat_back_addr": "10.20.0.11:6827\/452386", "heartbeat_front_addr": "10.20.0.11:6828\/452386", "state": [ "exists" ] }, { "osd": 42, "uuid": "15c862bb-fc1b-41b4-8f62-15772ac8c51a", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15732, "last_clean_end": 16133, "up_from": 17725, "up_thru": 17738, "down_at": 17754, "lost_at": 0, "public_addr": "10.20.0.11:6809\/454290", "cluster_addr": "10.20.0.11:6810\/454290", "heartbeat_back_addr": "10.20.0.11:6816\/454290", "heartbeat_front_addr": "10.20.0.11:6817\/454290", "state": [ "exists" ] }, { "osd": 43, "uuid": "ae8550c7-cd08-4d32-91ea-fe1e2909ec9e", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15727, "last_clean_end": 16135, "up_from": 17722, "up_thru": 17732, "down_at": 17755, "lost_at": 0, "public_addr": "10.20.0.11:6818\/454306", "cluster_addr": "10.20.0.11:6819\/454306", "heartbeat_back_addr": "10.20.0.11:6820\/454306", "heartbeat_front_addr": "10.20.0.11:6824\/454306", "state": [ "exists" ] }, { "osd": 44, "uuid": "a10860b0-6430-4f0f-ad10-27b87b29c202", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15733, "last_clean_end": 16141, "up_from": 17724, "up_thru": 17743, "down_at": 17756, "lost_at": 0, "public_addr": "10.20.0.11:6825\/454350", "cluster_addr": "10.20.0.11:6826\/454350", "heartbeat_back_addr": "10.20.0.11:6827\/454350", "heartbeat_front_addr": "10.20.0.11:6828\/454350", "state": [ "exists" ] }, { "osd": 45, "uuid": "2046b313-4aa8-4172-8e58-46cb2d00d049", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 17766, "last_clean_end": 17805, "up_from": 17814, "up_thru": 17780, "down_at": 17837, "lost_at": 0, "public_addr": "10.20.0.11:6809\/459655", "cluster_addr": "10.20.0.11:6816\/459655", "heartbeat_back_addr": "10.20.0.11:6817\/459655", "heartbeat_front_addr": "10.20.0.11:6818\/459655", "state": [ "exists" ] }, { "osd": 46, "uuid": "9339227e-6423-43ea-a137-5221e243ff96", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15730, "last_clean_end": 16150, "up_from": 17761, "up_thru": 17779, "down_at": 17807, "lost_at": 0, "public_addr": "10.20.0.11:6820\/456529", "cluster_addr": "10.20.0.11:6824\/456529", "heartbeat_back_addr": "10.20.0.11:6825\/456529", "heartbeat_front_addr": "10.20.0.11:6827\/456529", "state": [ "exists" ] }, { "osd": 47, "uuid": "ee4aa4f0-835e-4179-bcc9-a3b8029bda07", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 17765, "last_clean_end": 17807, "up_from": 17824, "up_thru": 17774, "down_at": 17838, "lost_at": 0, "public_addr": "10.20.0.11:6828\/460486", "cluster_addr": "10.20.0.11:6829\/460486", "heartbeat_back_addr": "10.20.0.11:6830\/460486", "heartbeat_front_addr": "10.20.0.11:6832\/460486", "state": [ "exists" ] }, { "osd": 48, "uuid": "80d1c85b-f6ad-4a53-8ff6-cb15e2b2c886", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 16556, "last_clean_end": 17679, "up_from": 17815, "up_thru": 16556, "down_at": 17833, "lost_at": 0, "public_addr": "10.20.0.12:6807\/182332", "cluster_addr": "10.20.0.12:6808\/182332", "heartbeat_back_addr": "10.20.0.12:6811\/182332", "heartbeat_front_addr": "10.20.0.12:6812\/182332", "state": [ "exists" ] }, { "osd": 49, "uuid": "3f5c4970-e8d5-446e-bedb-3c4364e06de8", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16129, "up_from": 17502, "up_thru": 16068, "down_at": 17681, "lost_at": 0, "public_addr": "10.20.0.12:6836\/175317", "cluster_addr": "10.20.0.12:6837\/175317", "heartbeat_back_addr": "10.20.0.12:6838\/175317", "heartbeat_front_addr": "10.20.0.12:6842\/175317", "state": [ "exists" ] }, { "osd": 50, "uuid": "43c6f777-42b9-418f-bbaf-afce89a1da33", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16130, "up_from": 17675, "up_thru": 16118, "down_at": 17682, "lost_at": 0, "public_addr": "10.20.0.12:6846\/176153", "cluster_addr": "10.20.0.12:6847\/176153", "heartbeat_back_addr": "10.20.0.12:6848\/176153", "heartbeat_front_addr": "10.20.0.12:6852\/176153", "state": [ "exists" ] }, { "osd": 51, "uuid": "04c9dc0c-836f-4828-8386-5390dc6312c9", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16131, "up_from": 17695, "up_thru": 16126, "down_at": 17714, "lost_at": 0, "public_addr": "10.20.0.12:6807\/176856", "cluster_addr": "10.20.0.12:6808\/176856", "heartbeat_back_addr": "10.20.0.12:6811\/176856", "heartbeat_front_addr": "10.20.0.12:6812\/176856", "state": [ "exists" ] }, { "osd": 52, "uuid": "035e9a74-d6ce-45d5-ba3c-1178f569f27b", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16134, "up_from": 17695, "up_thru": 16072, "down_at": 17715, "lost_at": 0, "public_addr": "10.20.0.12:6836\/176879", "cluster_addr": "10.20.0.12:6837\/176879", "heartbeat_back_addr": "10.20.0.12:6838\/176879", "heartbeat_front_addr": "10.20.0.12:6842\/176879", "state": [ "exists" ] }, { "osd": 53, "uuid": "808fb005-0ffd-4aba-b6ca-e3c8708af1f5", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 17695, "last_clean_end": 17716, "up_from": 17822, "up_thru": 16117, "down_at": 17834, "lost_at": 0, "public_addr": "10.20.0.12:6846\/182870", "cluster_addr": "10.20.0.12:6847\/182870", "heartbeat_back_addr": "10.20.0.12:6848\/182870", "heartbeat_front_addr": "10.20.0.12:6852\/182870", "state": [ "exists" ] }, { "osd": 54, "uuid": "b1d374ff-5d1b-4690-915f-8e6f1849ea79", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16139, "up_from": 17731, "up_thru": 17731, "down_at": 17751, "lost_at": 0, "public_addr": "10.20.0.12:6807\/178622", "cluster_addr": "10.20.0.12:6808\/178622", "heartbeat_back_addr": "10.20.0.12:6811\/178622", "heartbeat_front_addr": "10.20.0.12:6812\/178622", "state": [ "exists" ] }, { "osd": 55, "uuid": "f6c90ac8-a30a-4a37-8c85-5270fa8b423e", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16144, "up_from": 17737, "up_thru": 16129, "down_at": 17753, "lost_at": 0, "public_addr": "10.20.0.12:6836\/178648", "cluster_addr": "10.20.0.12:6838\/178648", "heartbeat_back_addr": "10.20.0.12:6842\/178648", "heartbeat_front_addr": "10.20.0.12:6846\/178648", "state": [ "exists" ] }, { "osd": 56, "uuid": "6d5a3648-bb8d-4aa7-9686-16885790c5f8", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 17742, "last_clean_end": 17756, "up_from": 17820, "up_thru": 16116, "down_at": 17836, "lost_at": 0, "public_addr": "10.20.0.12:6836\/182573", "cluster_addr": "10.20.0.12:6837\/182573", "heartbeat_back_addr": "10.20.0.12:6838\/182573", "heartbeat_front_addr": "10.20.0.12:6842\/182573", "state": [ "exists" ] }, { "osd": 57, "uuid": "63338d14-910e-4d77-86c6-bdfde643736b", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16154, "up_from": 17769, "up_thru": 16139, "down_at": 17808, "lost_at": 0, "public_addr": "10.20.0.12:6807\/180329", "cluster_addr": "10.20.0.12:6811\/180329", "heartbeat_back_addr": "10.20.0.12:6812\/180329", "heartbeat_front_addr": "10.20.0.12:6836\/180329", "state": [ "exists" ] }, { "osd": 58, "uuid": "13199a09-0db2-4ac5-9d58-3d633b93d14f", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15783, "last_clean_end": 16159, "up_from": 17772, "up_thru": 17777, "down_at": 17810, "lost_at": 0, "public_addr": "10.20.0.12:6837\/180352", "cluster_addr": "10.20.0.12:6842\/180352", "heartbeat_back_addr": "10.20.0.12:6846\/180352", "heartbeat_front_addr": "10.20.0.12:6847\/180352", "state": [ "exists" ] }, { "osd": 59, "uuid": "15723089-c846-4d6d-bf23-ee9ea418ba21", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 16566, "last_clean_end": 17682, "up_from": 17826, "up_thru": 17353, "down_at": 17831, "lost_at": 0, "public_addr": "10.20.0.13:6807\/416922", "cluster_addr": "10.20.0.13:6808\/416922", "heartbeat_back_addr": "10.20.0.13:6809\/416922", "heartbeat_front_addr": "10.20.0.13:6811\/416922", "state": [ "exists" ] }, { "osd": 60, "uuid": "7d768064-4ab0-4962-a95e-1257ffa839eb", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15683, "last_clean_end": 16131, "up_from": 17662, "up_thru": 16066, "down_at": 17685, "lost_at": 0, "public_addr": "10.20.0.13:6807\/406378", "cluster_addr": "10.20.0.13:6808\/406378", "heartbeat_back_addr": "10.20.0.13:6809\/406378", "heartbeat_front_addr": "10.20.0.13:6811\/406378", "state": [ "exists" ] }, { "osd": 61, "uuid": "576d436b-19d2-4eb1-ae6e-0f19a52c78e6", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 17676, "last_clean_end": 17685, "up_from": 17823, "up_thru": 16080, "down_at": 17832, "lost_at": 0, "public_addr": "10.20.0.13:6802\/416491", "cluster_addr": "10.20.0.13:6804\/416491", "heartbeat_back_addr": "10.20.0.13:6805\/416491", "heartbeat_front_addr": "10.20.0.13:6806\/416491", "state": [ "exists" ] }, { "osd": 62, "uuid": "5d1a9351-a39d-43c3-8945-6bd551ecb988", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15682, "last_clean_end": 16133, "up_from": 17693, "up_thru": 16121, "down_at": 17719, "lost_at": 0, "public_addr": "10.20.0.13:6802\/407884", "cluster_addr": "10.20.0.13:6804\/407884", "heartbeat_back_addr": "10.20.0.13:6805\/407884", "heartbeat_front_addr": "10.20.0.13:6806\/407884", "state": [ "exists" ] }, { "osd": 63, "uuid": "ad840dee-a8f7-4074-9f53-729075712b62", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15679, "last_clean_end": 16141, "up_from": 17698, "up_thru": 16079, "down_at": 17720, "lost_at": 0, "public_addr": "10.20.0.13:6807\/407910", "cluster_addr": "10.20.0.13:6808\/407910", "heartbeat_back_addr": "10.20.0.13:6809\/407910", "heartbeat_front_addr": "10.20.0.13:6811\/407910", "state": [ "exists" ] }, { "osd": 64, "uuid": "a8b2d98a-3be7-4bfe-9187-e1fe77a2b179", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15679, "last_clean_end": 16143, "up_from": 17697, "up_thru": 16140, "down_at": 17721, "lost_at": 0, "public_addr": "10.20.0.13:6812\/407963", "cluster_addr": "10.20.0.13:6813\/407963", "heartbeat_back_addr": "10.20.0.13:6814\/407963", "heartbeat_front_addr": "10.20.0.13:6816\/407963", "state": [ "exists" ] }, { "osd": 65, "uuid": "eab0c4b3-bc56-4157-b60a-51f7b7ac4732", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15675, "last_clean_end": 16148, "up_from": 17725, "up_thru": 16125, "down_at": 17747, "lost_at": 0, "public_addr": "10.20.0.13:6802\/410040", "cluster_addr": "10.20.0.13:6805\/410040", "heartbeat_back_addr": "10.20.0.13:6806\/410040", "heartbeat_front_addr": "10.20.0.13:6807\/410040", "state": [ "exists" ] }, { "osd": 66, "uuid": "227736ac-0a48-4b7a-936b-1a9d241c3c99", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15675, "last_clean_end": 16153, "up_from": 17727, "up_thru": 16128, "down_at": 17748, "lost_at": 0, "public_addr": "10.20.0.13:6809\/410063", "cluster_addr": "10.20.0.13:6811\/410063", "heartbeat_back_addr": "10.20.0.13:6812\/410063", "heartbeat_front_addr": "10.20.0.13:6814\/410063", "state": [ "exists" ] }, { "osd": 67, "uuid": "a8c1b84a-ef5a-47ff-ad49-0c3316b97bf6", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15675, "last_clean_end": 16158, "up_from": 17723, "up_thru": 16089, "down_at": 17749, "lost_at": 0, "public_addr": "10.20.0.13:6816\/410115", "cluster_addr": "10.20.0.13:6817\/410115", "heartbeat_back_addr": "10.20.0.13:6818\/410115", "heartbeat_front_addr": "10.20.0.13:6819\/410115", "state": [ "exists" ] }, { "osd": 68, "uuid": "0d9a5e37-cc79-46a7-89f6-b5e489dfb23e", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15678, "last_clean_end": 16162, "up_from": 17760, "up_thru": 16070, "down_at": 17811, "lost_at": 0, "public_addr": "10.20.0.13:6802\/412386", "cluster_addr": "10.20.0.13:6804\/412386", "heartbeat_back_addr": "10.20.0.13:6805\/412386", "heartbeat_front_addr": "10.20.0.13:6806\/412386", "state": [ "exists" ] }, { "osd": 69, "uuid": "ad69308b-7562-4880-b73c-ca05a2d8fc2e", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15678, "last_clean_end": 16167, "up_from": 17759, "up_thru": 16072, "down_at": 17812, "lost_at": 0, "public_addr": "10.20.0.13:6807\/412409", "cluster_addr": "10.20.0.13:6808\/412409", "heartbeat_back_addr": "10.20.0.13:6809\/412409", "heartbeat_front_addr": "10.20.0.13:6812\/412409", "state": [ "exists" ] }, { "osd": 70, "uuid": "d4b7ba97-3298-4fd0-9b4a-e461e88352de", "up": 0, "in": 0, "weight": 0.000000, "primary_affinity": 1.000000, "last_clean_begin": 15674, "last_clean_end": 16172, "up_from": 17761, "up_thru": 17768, "down_at": 17813, "lost_at": 0, "public_addr": "10.20.0.13:6813\/412461", "cluster_addr": "10.20.0.13:6814\/412461", "heartbeat_back_addr": "10.20.0.13:6816\/412461", "heartbeat_front_addr": "10.20.0.13:6817\/412461", "state": [ "exists" ] } ], "osd_xinfo": [ { "osd": 0, "down_stamp": "2015-04-26 02:00:19.572511", "laggy_probability": 0.927101, "laggy_interval": 46, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 1, "down_stamp": "2015-04-26 02:00:19.572511", "laggy_probability": 0.896128, "laggy_interval": 43, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 2, "down_stamp": "2015-04-26 02:00:19.572511", "laggy_probability": 0.882482, "laggy_interval": 68, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 3, "down_stamp": "2015-04-26 02:21:39.868625", "laggy_probability": 0.943535, "laggy_interval": 41, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 4, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.917737, "laggy_interval": 56, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 5, "down_stamp": "2015-04-26 02:02:17.842800", "laggy_probability": 0.882482, "laggy_interval": 15, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 6, "down_stamp": "2015-04-26 02:00:19.572511", "laggy_probability": 0.917737, "laggy_interval": 59, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 7, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.917737, "laggy_interval": 95, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 8, "down_stamp": "2015-04-26 04:36:29.410592", "laggy_probability": 0.642416, "laggy_interval": 58, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 9, "down_stamp": "2015-04-26 02:00:19.572511", "laggy_probability": 0.919336, "laggy_interval": 24, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 10, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.880998, "laggy_interval": 69, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 11, "down_stamp": "2015-04-26 04:36:35.768784", "laggy_probability": 0.604091, "laggy_interval": 32, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 12, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.849601, "laggy_interval": 59, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 13, "down_stamp": "2015-04-26 03:22:55.024663", "laggy_probability": 0.859651, "laggy_interval": 42, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 14, "down_stamp": "2015-04-26 03:47:02.193069", "laggy_probability": 0.948316, "laggy_interval": 30, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 15, "down_stamp": "2015-04-26 03:22:33.512773", "laggy_probability": 0.945507, "laggy_interval": 26, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 16, "down_stamp": "2015-04-26 03:22:20.137195", "laggy_probability": 0.948413, "laggy_interval": 19, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 17, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.773384, "laggy_interval": 72, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 18, "down_stamp": "2015-04-26 03:22:45.016578", "laggy_probability": 0.889015, "laggy_interval": 50, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 19, "down_stamp": "2015-04-26 03:22:33.512773", "laggy_probability": 0.931228, "laggy_interval": 29, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 20, "down_stamp": "2015-04-26 03:22:55.024663", "laggy_probability": 0.894947, "laggy_interval": 43, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 21, "down_stamp": "2015-04-26 03:22:41.016805", "laggy_probability": 0.933753, "laggy_interval": 14, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 22, "down_stamp": "2015-04-26 03:47:12.044546", "laggy_probability": 0.951671, "laggy_interval": 18, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 23, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.849840, "laggy_interval": 65, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 24, "down_stamp": "2015-04-26 02:03:26.390577", "laggy_probability": 0.806616, "laggy_interval": 47, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 25, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.810979, "laggy_interval": 52, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 26, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.858681, "laggy_interval": 132, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 27, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.796432, "laggy_interval": 70, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 28, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.908335, "laggy_interval": 39, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 29, "down_stamp": "2015-04-26 02:03:26.390577", "laggy_probability": 0.732148, "laggy_interval": 53, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 30, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.866609, "laggy_interval": 44, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 31, "down_stamp": "2015-04-26 02:21:39.868625", "laggy_probability": 0.866052, "laggy_interval": 46, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 32, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.812573, "laggy_interval": 53, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 33, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.797734, "laggy_interval": 62, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 34, "down_stamp": "2015-04-26 02:21:21.502025", "laggy_probability": 0.681026, "laggy_interval": 108, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 35, "down_stamp": "2015-04-26 02:03:26.390577", "laggy_probability": 0.703038, "laggy_interval": 47, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 36, "down_stamp": "2015-04-26 04:26:34.625477", "laggy_probability": 0.583865, "laggy_interval": 14, "features": 1125899906842623, "old_weight": 0 }, { "osd": 37, "down_stamp": "2015-04-26 04:26:37.902343", "laggy_probability": 0.585348, "laggy_interval": 35, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 38, "down_stamp": "2015-04-26 04:26:41.237620", "laggy_probability": 0.561149, "laggy_interval": 37, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 39, "down_stamp": "2015-04-26 04:41:21.591738", "laggy_probability": 0.433094, "laggy_interval": 29, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 40, "down_stamp": "2015-04-26 04:31:10.523987", "laggy_probability": 0.641786, "laggy_interval": 19, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 41, "down_stamp": "2015-04-26 04:31:13.837051", "laggy_probability": 0.581195, "laggy_interval": 32, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 42, "down_stamp": "2015-04-26 04:34:14.780969", "laggy_probability": 0.618319, "laggy_interval": 45, "features": 1125899906842623, "old_weight": 0 }, { "osd": 43, "down_stamp": "2015-04-26 04:34:17.897798", "laggy_probability": 0.618319, "laggy_interval": 14, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 44, "down_stamp": "2015-04-26 04:34:20.994697", "laggy_probability": 0.617709, "laggy_interval": 37, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 45, "down_stamp": "2015-04-26 04:41:24.759914", "laggy_probability": 0.318740, "laggy_interval": 45, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 46, "down_stamp": "2015-04-26 04:37:59.919908", "laggy_probability": 0.583313, "laggy_interval": 62, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 47, "down_stamp": "2015-04-26 04:41:27.882318", "laggy_probability": 0.368083, "laggy_interval": 24, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 48, "down_stamp": "2015-04-26 04:41:16.379505", "laggy_probability": 0.376217, "laggy_interval": 36, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 49, "down_stamp": "2015-04-26 04:27:03.567206", "laggy_probability": 0.592149, "laggy_interval": 25, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 50, "down_stamp": "2015-04-26 04:27:04.676795", "laggy_probability": 0.555430, "laggy_interval": 43, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 51, "down_stamp": "2015-04-26 04:31:16.034637", "laggy_probability": 0.496930, "laggy_interval": 21, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 52, "down_stamp": "2015-04-26 04:31:19.128934", "laggy_probability": 0.587214, "laggy_interval": 22, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 53, "down_stamp": "2015-04-26 04:41:19.500570", "laggy_probability": 0.413507, "laggy_interval": 16, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 54, "down_stamp": "2015-04-26 04:34:04.259736", "laggy_probability": 0.584793, "laggy_interval": 19, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 55, "down_stamp": "2015-04-26 04:34:07.571042", "laggy_probability": 0.555430, "laggy_interval": 20, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 56, "down_stamp": "2015-04-26 04:41:22.650530", "laggy_probability": 0.382869, "laggy_interval": 13, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 57, "down_stamp": "2015-04-26 04:38:03.048749", "laggy_probability": 0.490565, "laggy_interval": 32, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 58, "down_stamp": "2015-04-26 04:38:06.220080", "laggy_probability": 0.546174, "laggy_interval": 46, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 59, "down_stamp": "2015-04-26 04:41:11.190236", "laggy_probability": 0.306253, "laggy_interval": 14, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 60, "down_stamp": "2015-04-26 04:27:18.492423", "laggy_probability": 0.554090, "laggy_interval": 17, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 61, "down_stamp": "2015-04-26 04:41:14.304955", "laggy_probability": 0.417258, "laggy_interval": 12, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 62, "down_stamp": "2015-04-26 04:31:35.273855", "laggy_probability": 0.597863, "laggy_interval": 12, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 63, "down_stamp": "2015-04-26 04:31:36.357348", "laggy_probability": 0.545616, "laggy_interval": 17, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 64, "down_stamp": "2015-04-26 04:31:39.476062", "laggy_probability": 0.554090, "laggy_interval": 31, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 65, "down_stamp": "2015-04-26 04:33:52.803421", "laggy_probability": 0.554090, "laggy_interval": 32, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 66, "down_stamp": "2015-04-26 04:33:55.914944", "laggy_probability": 0.554090, "laggy_interval": 48, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 67, "down_stamp": "2015-04-26 04:33:59.049520", "laggy_probability": 0.597863, "laggy_interval": 19, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 68, "down_stamp": "2015-04-26 04:38:11.822916", "laggy_probability": 0.554090, "laggy_interval": 16, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 69, "down_stamp": "2015-04-26 04:38:14.921856", "laggy_probability": 0.554090, "laggy_interval": 22, "features": 1125899906842623, "old_weight": 65536 }, { "osd": 70, "down_stamp": "2015-04-26 04:38:18.015390", "laggy_probability": 0.554090, "laggy_interval": 35, "features": 1125899906842623, "old_weight": 65536 } ], "pg_temp": [], "primary_temp": [], "blacklist": [], "erasure_code_profiles": { "default": { "directory": "\/usr\/lib\/ceph\/erasure-code", "k": "2", "m": "1", "plugin": "jerasure", "technique": "reed_sol_van" } } }
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com