Hi, I am trying to understand how the PG peering count could increase between two OSDMAP epochs when none of the OSDs went down or up, nor were there changes in their weights. This behaviour was seen on a Hammer cluster, but I guess it should be no different from the recent releases. I'm pasting the relevant piece of monitor logs below and attaching the relevant odmap dumps. As it can be seen, the changes to the osdmap between dump_e1053237 and dump_e10532398 seem to be restricted to a few pg_temp additions and changes to up_thru, but the OSD weights and OSD up/down state have remained the same. Kindly let me know what I'm missing here in my understanding. Thanks, K.Prasad Monitor logs -------------------- 2020-04-15 08:06:10.956681 mon.1 10.33.89.159:6789/0 52232955 : cluster [INF] pgmap v107549051: 18432 pgs: 18332 active+clean, 1 peering, 99 active+remapped+backfill_toofull; 330 TB data, 518 TB used, 596 TB / 1114 TB avail; 34776/1684702815 objects degraded (0.002%); 1766213/1684702815 objects misplaced (0.105%) 2020-04-15 08:06:11.862613 mon.1 10.33.89.159:6789/0 52232956 : cluster [INF] osdmap e1053236: 388 osds: 315 up, 308 in 2020-04-15 08:06:11.868380 mon.1 10.33.89.159:6789/0 52232957 : cluster [INF] pgmap v107549052: 18432 pgs: 18332 active+clean, 1 peering, 99 active+remapped+backfill_toofull; 330 TB data, 518 TB used, 596 TB / 1114 TB avail; 34776/1684702815 objects degraded (0.002%); 1766213/1684702815 objects misplaced (0.105%) 2020-04-15 08:06:10.460979 osd.57 10.33.249.161:6811/4066471 200448 : cluster [INF] 5.356s0 restarting backfill on osd.382(0) from (936954'62108,945456'62308] 0//0//-1 to 1053229'64769 2020-04-15 08:06:12.696839 mon.1 10.33.89.159:6789/0 52232959 : cluster [INF] osdmap e1053237: 388 osds: 315 up, 308 in 2020-04-15 08:06:12.703424 mon.1 10.33.89.159:6789/0 52232960 : cluster [INF] pgmap v107549053: 18432 pgs: 18332 active+clean, 1 peering, 99 active+remapped+backfill_toofull; 330 TB data, 518 TB used, 596 TB / 1114 TB avail; 34776/1684702815 objects degraded (0.002%); 1766213/1684702815 objects misplaced (0.105%) 2020-04-15 08:06:13.745554 mon.1 10.33.89.159:6789/0 52232961 : cluster [INF] osdmap e1053238: 388 osds: 315 up, 308 in 2020-04-15 08:06:13.899273 mon.1 10.33.89.159:6789/0 52232962 : cluster [INF] pgmap v107549054: 18432 pgs: 1 inactive, 1 active+remapped+wait_backfill+backfill_toofull, 2 activating, 17078 active+clean, 1251 peering, 2 activating+remapped, 5 remapped+peering, 1 activating+degraded, 91 active+remapped+backfill_toofull; 329 TB data, 518 TB used, 600 TB / 1118 TB avail; 32298/1678972195 objects degraded (0.002%); 1666716/1678972195 objects misplaced (0.099%) 2020-04-15 08:06:14.766039 mon.1 10.33.89.159:6789/0 52232963 : cluster [INF] osdmap e1053239: 388 osds: 315 up, 308 in 2020-04-15 08:06:14.833707 mon.1 10.33.89.159:6789/0 52232964 : cluster [INF] pgmap v107549055: 18432 pgs: 2 inactive, 1 active+remapped+wait_backfill+backfill_toofull, 3 activating, 16496 active+clean, 1794 peering, 4 activating+remapped, 42 remapped+peering, 3 activating+degraded, 87 active+remapped+backfill_toofull; 329 TB data, 518 TB used, 600 TB / 1118 TB avail; 30842/1678959760 objects degraded (0.002%); 1576530/1678959760 objects misplaced (0.094%) 2020-04-15 08:06:15.730301 mon.1 10.33.89.159:6789/0 52232965 : cluster [INF] osdmap e1053240: 388 osds: 315 up, 308 in 2020-04-15 08:06:15.774021 mon.1 10.33.89.159:6789/0 52232966 : cluster [INF] pgmap v107549056: 18432 pgs: 2 inactive, 1 active+remapped+wait_backfill+backfill_toofull, 3 activating, 16139 active+clean, 2103 peering, 6 activating+remapped, 92 remapped+peering, 3 activating+degraded, 83 active+remapped+backfill_toofull; 329 TB data, 518 TB used, 600 TB / 1118 TB avail; 29414/1678947337 objects degraded (0.002%); 1490753/1678947337 objects misplaced (0.089%) 2020-04-15 08:06:09.424940 osd.146 10.33.129.28:6800/2689 125912 : cluster [INF] 5.252fs0 restarting backfill on osd.382(3) from (935430'62507,945456'62707] 0//0//-1 to 1053229'65086 2020-04-15 08:06:16.713440 mon.1 10.33.89.159:6789/0 52232967 : cluster [INF] osdmap e1053241: 388 osds: 315 up, 308 in -- *-----------------------------------------------------------------------------------------* *This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.***** **** *Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.***** **** *Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.* _-----------------------------------------------------------------------------------------_ _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |