Re: Many errors about PG deviate more than 30% on a new cluster deployed by cephadm

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

thanks for your reply

No did not stop autoscaler

root@store-par2-node01:/home/user# ceph osd pool autoscale-status
POOL               SIZE  TARGET SIZE                RATE  RAW CAPACITY   RATIO  TARGET RATIO  EFFECTIVE RATIO  BIAS  PG_NUM  NEW PG_NUM  AUTOSCALE  BULK
.mgr             15488k                              3.0         2763T  0.0000                                  1.0       1              on         False
cephfs_data          0                1.3333333730697632         2763T  0.0000                                  1.0      32              on         False
cephfs_metadata   5742                               3.0         2763T  0.0000                                  4.0      32              on         False

I not in production, I can destroy the pool and recreate it, or destroy the cluster and rebuild it


----- Mail original -----
> De: "Eugen Block" <eblock@xxxxxx>
> À: "ceph-users" <ceph-users@xxxxxxx>
> Envoyé: Mardi 7 Juin 2022 15:00:39
> Objet:  Re: Many errors about PG deviate more than 30% on a new cluster deployed by cephadm

> Hi,
> 
> please share the output of 'ceph osd pool autoscale-status'. You have
> very low (too low) PG numbers per OSD (between 0 and 6), did you stop
> the autoscaler at an early stage? If you don't want to use the
> autoscaler you should increase the pg_num, but you could set
> autoscaler to warn mode and see what it suggests.
> 
> 
> Zitat von Christophe BAILLON <cb@xxxxxxx>:
> 
>> Hi all
>>
>> I got many error about PG deviation more than 30% on a new installed cluster
>>
>> This cluster is managed by cephadm
>>
>> all box 15 have :
>> 12 x 18Tb
>> 2 x nvme
>> 2 x ssd for boot
>>
>> Our main pool is on EC 6 + 2 for exclusive use with cephfs
>> created with this method :
>> ceph orch apply -i osd_spec.yaml
>>
>> with this conf
>>
>> osd_spec.yaml
>> service_type: osd
>> service_id: osd_spec_default
>> placement:
>>   host_pattern: '*'
>> data_devices:
>>   rotational: 1
>> db_devices:
>>   paths:
>>     - /dev/nvme0n1
>>     - /dev/nvme1n1
>>
>> root@store-par2-node01:/home/user# ceph -s
>>   cluster:
>>     id:     cf37418e-e0b9-11ec-95a5-f1f73d5801cb
>>     health: HEALTH_OK
>>
>>   services:
>>     mon: 5 daemons, quorum
>> store-par2-node01,store-par2-node02,store-par2-node03,store-par2-node04,store-par2-node05
>> (age
>> 5d)
>>     mgr: store-par2-node02.osbvrb(active, since 3d), standbys:
>> store-par2-node01.oerpqs
>>     mds: 1/1 daemons up, 3 standby
>>     osd: 168 osds: 168 up (since 3d), 168 in (since 5d)
>>
>>   data:
>>     volumes: 1/1 healthy
>>     pools:   3 pools, 65 pgs
>>     objects: 27 objects, 14 MiB
>>     usage:   13 TiB used, 2.7 PiB / 2.7 PiB avail
>>     pgs:     65 active+clean
>>
>> root@store-par2-node01:/home/user# ceph osd df tree
>> ID   CLASS  WEIGHT      REWEIGHT  SIZE     RAW USE  DATA     OMAP
>>  META     AVAIL    %USE  VAR   PGS  STATUS  TYPE NAME
>>  -1         2763.38159         -  2.7 PiB   13 TiB  9.6 GiB    9 KiB
>>  9.7 GiB  2.7 PiB  0.47  1.00    -          root default
>>  -3          197.38440         -  197 TiB  955 GiB  696 MiB    7 KiB
>>  608 MiB  196 TiB  0.47  1.00    -              host store-par2-node01
>>   0    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   60 MiB   16 TiB  0.47  1.00    2      up          osd.0
>>   1    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   55 MiB   16 TiB  0.47  1.00    1      up          osd.1
>>   2    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   55 MiB   16 TiB  0.47  1.00    1      up          osd.2
>>   3    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   46 MiB   16 TiB  0.47  1.00    5      up          osd.3
>>   4    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   45 MiB   16 TiB  0.47  1.00    2      up          osd.4
>>   5    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   45 MiB   16 TiB  0.47  1.00    1      up          osd.5
>>   6    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   45 MiB   16 TiB  0.47  1.00    0      up          osd.6
>>   7    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   46 MiB   16 TiB  0.47  1.00    1      up          osd.7
>>   8    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   47 MiB   16 TiB  0.47  1.00    4      up          osd.8
>>   9    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   49 MiB   16 TiB  0.47  1.00    2      up          osd.9
>>  10    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   55 MiB   16 TiB  0.47  1.00    4      up          osd.10
>>  11    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   60 MiB   16 TiB  0.47  1.00    3      up          osd.11
>>  -7          197.38440         -  197 TiB  955 GiB  696 MiB    2 KiB
>>  625 MiB  196 TiB  0.47  1.00    -              host store-par2-node02
>>  13    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    5      up          osd.13
>>  15    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    2      up          osd.15
>>  17    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    3      up          osd.17
>>  19    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    2      up          osd.19
>>  21    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    3      up          osd.21
>>  23    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    1      up          osd.23
>>  25    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    3      up          osd.25
>>  27    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    1      up          osd.27
>>  29    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    1      up          osd.29
>>  31    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    4      up          osd.31
>>  33    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   43 MiB   16 TiB  0.47  1.00    2      up          osd.33
>>  35    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB    1 KiB
>>   42 MiB   16 TiB  0.47  1.00    2      up          osd.35
>>  -5          197.38440         -  197 TiB  955 GiB  710 MiB      0 B
>>  994 MiB  196 TiB  0.47  1.00    -              host store-par2-node03
>>  12    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    0      up          osd.12
>>  14    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    2      up          osd.14
>>  16    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    0      up          osd.16
>>  18    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    3      up          osd.18
>>  20    hdd    16.44870   1.00000   16 TiB   80 GiB   71 MiB      0 B
>>  400 MiB   16 TiB  0.47  1.00    4      up          osd.20
>>  22    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    0      up          osd.22
>>  24    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    3      up          osd.24
>>  26    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    2      up          osd.26
>>  28    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    4      up          osd.28
>>  30    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    3      up          osd.30
>>  32    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    1      up          osd.32
>>  34    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    4      up          osd.34
>> -29          197.38440         -  197 TiB  955 GiB  710 MiB      0 B
>>  978 MiB  196 TiB  0.47  1.00    -              host store-par2-node04
>>  38    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.38
>>  53    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.53
>>  65    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.65
>>  78    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.78
>>  89    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.89
>> 101    hdd    16.44870   1.00000   16 TiB   80 GiB   71 MiB      0 B
>>  395 MiB   16 TiB  0.47  1.00    3      up          osd.101
>> 112    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.112
>> 123    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.123
>> 134    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.134
>> 145    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.145
>> 156    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.156
>> 167    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.167
>> -11          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node05
>>  46    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.46
>>  51    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.51
>>  60    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.60
>>  70    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.70
>>  81    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.81
>>  92    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.92
>> 103    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.103
>> 114    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.114
>> 125    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.125
>> 136    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.136
>> 147    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    5      up          osd.147
>> 158    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.158
>> -13          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node06
>>  36    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.36
>>  47    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.47
>>  59    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.59
>>  69    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.69
>>  82    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.82
>>  94    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.94
>> 106    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.106
>> 115    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.115
>> 128    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.128
>> 137    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.137
>> 148    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.148
>> 159    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.159
>> -23          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node07
>>  41    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.41
>>  48    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.48
>>  58    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.58
>>  71    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.71
>>  83    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.83
>>  93    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.93
>> 105    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.105
>> 117    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.117
>> 129    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.129
>> 138    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.138
>> 150    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    5      up          osd.150
>> 163    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.163
>> -17          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node08
>>  43    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.43
>>  56    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.56
>>  68    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.68
>>  79    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.79
>>  90    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.90
>> 100    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.100
>> 111    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.111
>> 121    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.121
>> 132    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    7      up          osd.132
>> 143    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.143
>> 154    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.154
>> 165    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.165
>> -27          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node09
>>  37    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.37
>>  49    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.49
>>  63    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.63
>>  77    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.77
>>  88    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.88
>>  98    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.98
>> 110    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.110
>> 120    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.120
>> 131    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.131
>> 141    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.141
>> 152    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.152
>> 164    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.164
>>  -9          197.38440         -  197 TiB  955 GiB  710 MiB      0 B
>>  982 MiB  196 TiB  0.47  1.00    -              host store-par2-node10
>>  45    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.45
>>  54    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.54
>>  64    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.64
>>  74    hdd    16.44870   1.00000   16 TiB   80 GiB   71 MiB      0 B
>>  399 MiB   16 TiB  0.47  1.00    5      up          osd.74
>>  87    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.87
>>  97    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.97
>> 107    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.107
>> 116    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.116
>> 127    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.127
>> 142    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.142
>> 153    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.153
>> 161    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.161
>> -15          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  637 MiB  196 TiB  0.47  1.00    -              host store-par2-node11
>>  40    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   54 MiB   16 TiB  0.47  1.00    6      up          osd.40
>>  50    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.50
>>  61    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.61
>>  73    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.73
>>  84    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.84
>>  95    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.95
>> 104    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    5      up          osd.104
>> 118    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.118
>> 126    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.126
>> 139    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.139
>> 149    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.149
>> 160    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.160
>> -25          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node12
>>  39    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.39
>>  52    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.52
>>  66    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.66
>>  76    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.76
>>  86    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.86
>>  99    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.99
>> 109    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.109
>> 122    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.122
>> 133    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.133
>> 144    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.144
>> 155    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.155
>> 166    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.166
>> -19          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node13
>>  42    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.42
>>  55    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.55
>>  62    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.62
>>  72    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.72
>>  80    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.80
>>  91    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    5      up          osd.91
>> 102    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.102
>> 113    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.113
>> 124    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.124
>> 135    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.135
>> 146    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.146
>> 157    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.157
>> -21          197.38440         -  197 TiB  955 GiB  696 MiB      0 B
>>  636 MiB  196 TiB  0.47  1.00    -              host store-par2-node14
>>  44    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.44
>>  57    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.57
>>  67    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    4      up          osd.67
>>  75    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.75
>>  85    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.85
>>  96    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.96
>> 108    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.108
>> 119    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    1      up          osd.119
>> 130    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    3      up          osd.130
>> 140    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    0      up          osd.140
>> 151    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.151
>> 162    hdd    16.44870   1.00000   16 TiB   80 GiB   58 MiB      0 B
>>   53 MiB   16 TiB  0.47  1.00    2      up          osd.162
>>                            TOTAL  2.7 PiB   13 TiB  9.6 GiB  9.6 KiB
>>  9.7 GiB  2.7 PiB  0.47
>> MIN/MAX VAR: 1.00/1.00  STDDEV: 0
>>
>>
>> root@store-par2-node01:/home/user# ceph pg dump
>> version 140443
>> stamp 2022-06-06T12:27:35.382577+0000
>> last_osdmap_epoch 0
>> last_pg_scan 0
>> PG_STAT  OBJECTS  MISSING_ON_PRIMARY  DEGRADED  MISPLACED  UNFOUND
>> BYTES     OMAP_BYTES*  OMAP_KEYS*  LOG    DISK_LOG  STATE
>> STATE_STAMP                      VERSION     REPORTED    UP
>>                        UP_PRIMARY  ACTING
>>  ACTING_PRIMARY  LAST_SCRUB  SCRUB_STAMP
>> LAST_DEEP_SCRUB  DEEP_SCRUB_STAMP                 SNAPTRIMQ_LEN
>> LAST_SCRUB_DURATION  SCRUB_SCHEDULING
>>             OBJECTS_SCRUBBED  OBJECTS_TRIMMED
>> 3.f            1                   0         0          0        0
>>      22            0           0      2         2  active+clean
>> 2022-06-06T02:22:55.899870+0000       773'2    1191:813
>>         [112,132,142]         112                      [112,132,142]
>>             112       773'2  2022-06-06T02:22:55.899728+0000
>>    773'2  2022-06-03T11:08:04.230774+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T02:26:26.925460+0000
>>                 1                0
>> 2.e            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T18:09:54.859041+0000         0'0    1191:818
>> [8,3,44,13,64,67,132,31]           8
>> [8,3,44,13,64,67,132,31]               8         0'0
>> 2022-06-05T18:09:54.858891+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T02:48:10.931317+0000
>>       0                0
>> 3.c            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T22:21:13.363161+0000         0'0    1191:813
>>             [24,8,52]          24                          [24,8,52]
>>              24         0'0  2022-06-05T22:21:13.363041+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T05:12:34.695987+0000
>>                 0                0
>> 2.d            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T21:38:59.942761+0000         0'0    1191:819
>> [51,134,3,86,96,38,1,41]          51
>> [51,134,3,86,96,38,1,41]              51         0'0
>> 2022-06-05T21:38:59.942666+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T22:34:01.326089+0000
>>       0                0
>> 3.d            1                   0         0          0        0
>>      70            0           0      2         2  active+clean
>> 2022-06-06T06:48:17.160609+0000       771'2    1191:816
>>          [122,71,132]         122                       [122,71,132]
>>             122       771'2  2022-06-06T06:48:17.160469+0000
>>    771'2  2022-06-03T10:46:08.465824+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T10:22:35.365062+0000
>>                 1                0
>> 2.c            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T04:17:49.079390+0000         0'0    1191:819
>> [149,19,48,11,158,60,143,110]         149
>> [149,19,48,11,158,60,143,110]             149         0'0
>> 2022-06-06T04:17:49.079269+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T06:16:29.279337+0000
>>       0                0
>> 3.a            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T02:51:41.932102+0000         0'0    1191:813
>>          [18,148,131]          18                       [18,148,131]
>>              18         0'0  2022-06-06T02:51:41.931974+0000
>>      0'0  2022-06-04T15:55:16.683239+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T10:39:23.180043+0000
>>                 0                0
>> 2.b            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T10:08:33.237436+0000         0'0    1191:818
>> [46,159,28,74,137,37,91,127]          46
>> [46,159,28,74,137,37,91,127]              46         0'0
>> 2022-06-06T10:08:33.237340+0000              0'0
>> 2022-06-06T10:08:33.237340+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T10:13:44.177114+0000
>>       0                0
>> 3.b            1                   0         0          0        0
>>       0            0           0      3         3  active+clean
>> 2022-06-06T00:54:52.451573+0000       671'3    1191:814
>>          [73,159,147]          73                       [73,159,147]
>>              73       671'3  2022-06-06T00:54:52.451459+0000
>>    671'3  2022-06-03T03:41:08.026476+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T12:45:10.753628+0000
>>                 1                0
>> 2.a            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T22:53:09.039828+0000         0'0    1191:815
>> [44,86,132,94,71,81,80,166]          44
>> [44,86,132,94,71,81,80,166]              44         0'0
>> 2022-06-05T22:53:09.039734+0000              0'0
>> 2022-06-03T12:58:37.531167+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T03:20:37.899744+0000
>>       0                0
>> 3.8            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T14:40:36.607866+0000         0'0    1191:811
>>           [115,38,68]         115                        [115,38,68]
>>             115         0'0  2022-06-05T14:40:36.607760+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T14:45:02.993139+0000
>>                 0                0
>> 2.9            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T03:20:19.920382+0000         0'0    1191:817
>> [104,84,110,40,82,73,147,17]         104
>> [104,84,110,40,82,73,147,17]             104         0'0
>> 2022-06-06T03:20:19.920257+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T06:24:06.295277+0000
>>       0                0
>> 3.9            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T22:14:34.542404+0000         0'0    1191:811
>>           [59,45,129]          59                        [59,45,129]
>>              59         0'0  2022-06-05T22:14:34.542285+0000
>>      0'0  2022-06-03T00:14:09.631316+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T06:57:37.776582+0000
>>                 0                0
>> 2.8            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T22:12:11.172949+0000         0'0    1191:815
>> [45,101,104,13,143,157,63,61]          45
>> [45,101,104,13,143,157,63,61]              45         0'0
>> 2022-06-05T22:12:11.172853+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T06:52:53.792844+0000
>>       0                0
>> 3.6            1                   0         0          0        0
>>       0            0           0      5         5  active+clean
>> 2022-06-05T10:01:53.243050+0000      1043'5    1191:818
>>           [98,66,132]          98                        [98,66,132]
>>              98      1043'5  2022-06-05T10:01:53.242948+0000
>>   1043'5  2022-06-05T10:01:53.242948+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T18:00:59.024882+0000
>>                 1                0
>> 2.7            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T08:28:41.853562+0000         0'0    1191:818
>> [163,57,151,110,116,118,30,147]         163
>> [163,57,151,110,116,118,30,147]             163         0'0
>> 2022-06-06T08:28:41.853404+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T18:31:27.727201+0000
>>       0                0
>> 3.7            1                   0         0          0        0
>>       0            0           0      2         2  active+clean
>> 2022-06-06T11:35:38.186470+0000       437'2    1191:826
>>            [95,31,55]          95                         [95,31,55]
>>              95       437'2  2022-06-06T11:35:38.186365+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T12:34:45.361534+0000
>>                 1                0
>> 2.6            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T06:13:11.860365+0000         0'0    1191:817
>> [135,112,19,98,114,88,119,8]         135
>> [135,112,19,98,114,88,119,8]             135         0'0
>> 2022-06-06T06:13:11.860264+0000              0'0
>> 2022-06-06T06:13:11.860264+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T08:33:30.327642+0000
>>       0                0
>> 3.4            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T11:53:13.051840+0000         0'0    1191:813
>>           [10,13,131]          10                        [10,13,131]
>>              10         0'0  2022-06-05T11:53:13.051678+0000
>>      0'0  2022-06-02T02:18:09.375232+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T14:15:18.261225+0000
>>                 0                0
>> 2.5            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T15:15:43.099447+0000         0'0    1191:817
>> [15,4,8,144,81,96,73,10]          15
>> [15,4,8,144,81,96,73,10]              15         0'0
>> 2022-06-05T15:15:43.099314+0000              0'0
>> 2022-06-04T11:38:14.307983+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T22:44:13.570864+0000
>>       0                0
>> 2.0            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T23:46:06.362681+0000         0'0    1191:844
>> [30,109,13,40,133,136,37,65]          30
>> [30,109,13,40,133,136,37,65]              30         0'0
>> 2022-06-05T23:46:06.362567+0000              0'0
>> 2022-06-05T23:46:06.362567+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T05:50:48.092572+0000
>>       0                0
>> 3.1            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T16:32:59.830885+0000         0'0    1191:831
>>           [73,150,54]          73                        [73,150,54]
>>              73         0'0  2022-06-05T16:32:59.830683+0000
>>      0'0  2022-06-05T16:32:59.830683+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T02:38:21.783899+0000
>>                 0                0
>> 1.0            5                   0         0          0        0
>> 14156320            0           0  10005     10005  active+clean
>> 2022-06-05T16:52:10.117602+0000  1146'21705  1191:26731
>>           [74,101,20]          74                        [74,101,20]
>>              74  1012'18422  2022-06-05T16:52:10.117484+0000
>>      0'0  2022-05-31T17:19:28.164530+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T23:19:35.924762+0000
>>                 4                0
>> 2.3            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T15:59:18.303912+0000         0'0    1191:819
>> [83,121,122,34,92,21,32,72]          83
>> [83,121,122,34,92,21,32,72]              83         0'0
>> 2022-06-05T15:59:18.303794+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T02:34:48.910045+0000
>>       0                0
>> 3.2            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T03:07:55.854126+0000         0'0    1191:811
>>           [164,31,34]         164                        [164,31,34]
>>             164         0'0  2022-06-06T03:07:55.854014+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T08:11:53.316960+0000
>>                 0                0
>> 2.1            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T13:03:19.274658+0000         0'0    1191:821
>> [40,76,99,128,43,135,10,96]          40
>> [40,76,99,128,43,135,10,96]              40         0'0
>> 2022-06-05T13:03:19.274545+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T18:43:52.710554+0000
>>       0                0
>> 3.0            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T13:33:06.917207+0000         0'0    1191:826
>>            [35,40,70]          35                         [35,40,70]
>>              35         0'0  2022-06-05T13:33:06.917074+0000
>>      0'0  2022-05-31T19:52:08.234732+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T18:08:03.107087+0000
>>                 0                0
>> 3.3            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T11:43:56.320476+0000         0'0    1191:840
>>            [40,18,11]          40                         [40,18,11]
>>              40         0'0  2022-06-06T11:43:56.320361+0000
>>      0'0  2022-06-04T06:13:59.535590+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T20:55:33.787766+0000
>>                 0                0
>> 2.2            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T16:42:20.204103+0000         0'0    1191:819
>> [71,150,24,82,144,48,161,91]          71
>> [71,150,24,82,144,48,161,91]              71         0'0
>> 2022-06-05T16:42:20.203967+0000              0'0
>> 2022-06-03T04:54:17.344478+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T21:40:54.542796+0000
>>       0                0
>> 3.5            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T20:06:50.152236+0000         0'0    1191:812
>>            [152,3,93]         152                         [152,3,93]
>>             152         0'0  2022-06-05T20:06:50.152134+0000
>>      0'0  2022-06-03T08:27:56.053931+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T05:07:45.942271+0000
>>                 0                0
>> 2.4            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T23:44:27.330764+0000         0'0    1191:816
>> [11,132,21,7,104,149,49,134]          11
>> [11,132,21,7,104,149,49,134]              11         0'0
>> 2022-06-05T23:44:27.330642+0000              0'0
>> 2022-06-05T23:44:27.330642+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T03:48:00.509825+0000
>>       0                0
>> 3.e            1                   0         0          0        0
>>       0            0           0      3         3  active+clean
>> 2022-06-05T20:45:57.476116+0000       543'3    1191:814
>>            [37,97,26]          37                         [37,97,26]
>>              37       543'3  2022-06-05T20:45:57.476006+0000
>>    543'3  2022-06-02T00:50:17.270315+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T00:57:37.043277+0000
>>                 1                0
>> 2.f            0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T10:18:51.955868+0000         0'0    1191:827
>> [92,85,101,53,25,24,113,145]          92
>> [92,85,101,53,25,24,113,145]              92         0'0
>> 2022-06-06T10:18:51.955714+0000              0'0
>> 2022-06-05T06:18:02.624483+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T12:10:14.007061+0000
>>       0                0
>> 3.11           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T20:15:43.246605+0000         0'0    1191:811
>>            [33,30,41]          33                         [33,30,41]
>>              33         0'0  2022-06-05T20:15:43.246445+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T01:59:04.536885+0000
>>                 0                0
>> 2.10           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T03:41:55.049114+0000         0'0    1191:819
>> [130,167,43,143,66,65,45,145]         130
>> [130,167,43,143,66,65,45,145]             130         0'0
>> 2022-06-06T03:41:55.048887+0000              0'0
>> 2022-06-06T03:41:55.048887+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T12:59:20.841601+0000
>>       0                0
>> 3.10           4                   0         0          0        0
>>       0            0           0     12        12  active+clean
>> 2022-06-05T06:44:51.783819+0000      845'12    1191:816
>>          [130,103,93]         130                       [130,103,93]
>>             130      845'12  2022-06-05T06:44:51.783710+0000
>>   845'12  2022-06-03T19:13:59.475041+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T16:31:37.232782+0000
>>                 4                0
>> 2.11           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T08:38:43.779419+0000         0'0    1191:820
>> [107,20,129,122,27,113,25,83]         107
>> [107,20,129,122,27,113,25,83]             107         0'0
>> 2022-06-06T08:38:43.779269+0000              0'0
>> 2022-06-03T11:14:02.841282+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T16:53:01.369556+0000
>>       0                0
>> 3.13           2                   0         0          0        0
>>      34            0           0      2         2  active+clean
>> 2022-06-05T15:00:46.595842+0000       437'2    1191:816
>>           [149,14,85]         149                        [149,14,85]
>>             149       437'2  2022-06-05T15:00:46.595730+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T16:27:45.919806+0000
>>                 2                0
>> 2.12           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T07:10:51.207987+0000         0'0    1191:815
>> [35,139,147,106,60,28,126,127]          35
>> [35,139,147,106,60,28,126,127]              35         0'0
>> 2022-06-06T07:10:51.207861+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T17:59:37.937977+0000
>>       0                0
>> 3.12           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T09:38:30.626384+0000         0'0    1191:812
>>         [135,154,131]         135                      [135,154,131]
>>             135         0'0  2022-06-05T09:38:30.626285+0000
>>      0'0  2022-06-02T22:21:59.376573+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T18:43:10.579197+0000
>>                 0                0
>> 2.13           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T01:39:10.599882+0000         0'0    1191:818
>> [74,36,3,158,39,77,75,98]          74
>> [74,36,3,158,39,77,75,98]              74         0'0
>> 2022-06-06T01:39:10.599742+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T07:50:19.048812+0000
>>       0                0
>> 3.15           2                   0         0          0        0
>>      90            0           0      3         3  active+clean
>> 2022-06-06T00:06:19.711529+0000       437'3    1191:817
>>           [54,15,102]          54                        [54,15,102]
>>              54       437'3  2022-06-06T00:06:19.711396+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T09:05:26.475289+0000
>>                 2                0
>> 2.14           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T13:23:51.227730+0000         0'0    1191:815
>> [63,18,104,31,28,80,107,47]          63
>> [63,18,104,31,28,80,107,47]              63         0'0
>> 2022-06-05T13:23:51.227619+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T00:57:54.831856+0000
>>       0                0
>> 3.14           4                   0         0          0        0
>>    4972            0           0     37        37  active+clean
>> 2022-06-05T18:43:38.650054+0000      718'37    1191:851
>>          [83,114,153]          83                       [83,114,153]
>>              83      718'37  2022-06-05T18:43:38.649890+0000
>>   718'37  2022-06-05T18:43:38.649890+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T01:34:37.932141+0000
>>                 4                0
>> 2.15           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T16:15:50.881954+0000         0'0    1191:819
>> [54,88,142,33,76,20,146,150]          54
>> [54,88,142,33,76,20,146,150]              54         0'0
>> 2022-06-05T16:15:50.881848+0000              0'0
>> 2022-06-01T21:50:03.827583+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T16:24:10.967689+0000
>>       0                0
>> 3.17           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T16:14:32.120728+0000         0'0    1191:814
>>          [147,34,167]         147                       [147,34,167]
>>             147         0'0  2022-06-05T16:14:32.120636+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T01:17:17.942645+0000
>>                 0                0
>> 2.16           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T13:32:29.676884+0000         0'0    1191:815
>> [162,152,23,74,160,9,121,10]         162
>> [162,152,23,74,160,9,121,10]             162         0'0
>> 2022-06-05T13:32:29.676755+0000              0'0
>> 2022-06-01T23:18:50.309093+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T15:29:19.493434+0000
>>       0                0
>> 3.16           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T03:05:19.005785+0000         0'0    1191:813
>>          [26,162,105]          26                       [26,162,105]
>>              26         0'0  2022-06-06T03:05:19.005679+0000
>>      0'0  2022-05-31T19:52:13.903980+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T05:38:41.233380+0000
>>                 0                0
>> 2.17           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T16:25:32.999773+0000         0'0    1191:821
>> [146,104,158,163,108,106,141,159]         146
>> [146,104,158,163,108,106,141,159]             146         0'0
>> 2022-06-05T16:25:32.999664+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T03:44:47.468570+0000
>>       0                0
>> 3.19           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T18:18:12.164305+0000         0'0    1191:810
>>          [156,40,109]         156                       [156,40,109]
>>             156         0'0  2022-06-05T18:18:12.164187+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T20:18:30.481546+0000
>>                 0                0
>> 2.18           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T18:42:00.227559+0000         0'0    1191:820
>> [107,105,97,17,99,124,154,132]         107
>> [107,105,97,17,99,124,154,132]             107         0'0
>> 2022-06-05T18:42:00.227444+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T00:48:30.997533+0000
>>       0                0
>> 3.18           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T22:29:35.968669+0000         0'0    1191:813
>>           [150,21,36]         150                        [150,21,36]
>>             150         0'0  2022-06-05T22:29:35.968576+0000
>>      0'0  2022-06-01T20:07:31.626680+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T08:00:54.614246+0000
>>                 0                0
>> 2.19           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T12:25:06.541967+0000         0'0    1192:820
>> [131,151,63,13,146,90,135,133]         131
>> [131,151,63,13,146,90,135,133]             131         0'0
>> 2022-06-06T12:25:06.541855+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T16:49:45.723687+0000
>>       0                0
>> 3.1b           0                   0         0          0        0
>>       0            0           0      1         1  active+clean
>> 2022-06-05T12:05:36.929834+0000       437'1    1191:813
>>          [91,120,142]          91                       [91,120,142]
>>              91       437'1  2022-06-05T12:05:36.929706+0000
>>    437'1  2022-06-02T00:32:32.667254+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T20:33:52.594948+0000
>>                 0                0
>> 2.1a           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T21:33:42.799425+0000         0'0    1191:817
>> [58,112,85,50,129,148,49,45]          58
>> [58,112,85,50,129,148,49,45]              58         0'0
>> 2022-06-05T21:33:42.799305+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T08:52:06.385325+0000
>>       0                0
>> 3.1a           0                   0         0          0        0
>>       0            0           0      1         1  active+clean
>> 2022-06-05T14:57:02.326186+0000       437'1    1191:815
>>             [25,9,93]          25                          [25,9,93]
>>              25       437'1  2022-06-05T14:57:02.326068+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T22:00:49.835756+0000
>>                 0                0
>> 2.1b           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T00:21:22.013136+0000         0'0    1191:819
>> [51,42,60,83,164,145,52,134]          51
>> [51,42,60,83,164,145,52,134]              51         0'0
>> 2022-06-06T00:21:22.013028+0000              0'0
>> 2022-06-06T00:21:22.013028+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T08:38:12.431836+0000
>>       0                0
>> 3.1d           1                   0         0          0        0
>>       0            0           0      2         2  active+clean
>> 2022-06-06T00:17:33.395234+0000       437'2    1191:815
>>           [150,67,20]         150                        [150,67,20]
>>             150       437'2  2022-06-06T00:17:33.395098+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T05:38:06.038546+0000
>>                 1                0
>> 2.1c           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T21:38:58.966683+0000         0'0    1191:819
>> [51,34,80,144,143,99,50,0]          51
>> [51,34,80,144,143,99,50,0]              51         0'0
>> 2022-06-05T21:38:58.966531+0000              0'0
>> 2022-06-03T07:43:07.143846+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T04:01:52.286592+0000
>>       0                0
>> 3.1c           1                   0         0          0        0
>>       0            0           0      3         3  active+clean
>> 2022-06-06T05:23:02.301909+0000       724'3    1191:815
>>          [76,156,128]          76                       [76,156,128]
>>              76       724'3  2022-06-06T05:23:02.301761+0000
>>    724'3  2022-06-03T07:49:44.813182+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T11:37:54.326001+0000
>>                 1                0
>> 2.1d           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-06T03:14:29.829206+0000         0'0    1191:816
>> [67,86,61,81,130,69,74,46]          67
>> [67,86,61,81,130,69,74,46]              67         0'0
>> 2022-06-06T03:14:29.829088+0000              0'0
>> 2022-06-03T06:14:14.581690+0000              0                    1
>> periodic scrub scheduled @ 2022-06-07T12:37:45.026826+0000
>>       0                0
>> 3.1f           2                   0         0          0        0
>>     554            0           0      4         4  active+clean
>> 2022-06-05T13:34:03.534647+0000       437'4    1191:815
>>          [160,112,49]         160                       [160,112,49]
>>             160       437'4  2022-06-05T13:34:03.534531+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-06T15:30:53.351210+0000
>>                 2                0
>> 2.1e           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T16:05:18.546234+0000         0'0    1191:817
>> [5,0,91,118,2,107,41,37]           5
>> [5,0,91,118,2,107,41,37]               5         0'0
>> 2022-06-05T16:05:18.546142+0000              0'0
>> 2022-05-31T19:51:41.372150+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T23:22:10.205148+0000
>>       0                0
>> 3.1e           0                   0         0          0        0
>>       0            0           0      1         1  active+clean
>> 2022-06-06T02:05:23.255525+0000       437'1    1191:814
>>           [3,166,110]           3                        [3,166,110]
>>               3       437'1  2022-06-06T02:05:23.255430+0000
>>      0'0  2022-05-31T19:51:59.873880+0000              0
>>        1  periodic scrub scheduled @ 2022-06-07T12:43:16.169680+0000
>>                 0                0
>> 2.1f           0                   0         0          0        0
>>       0            0           0      0         0  active+clean
>> 2022-06-05T14:19:38.386679+0000         0'0    1191:816
>> [67,14,117,29,28,4,91,17]          67
>> [67,14,117,29,28,4,91,17]              67         0'0
>> 2022-06-05T14:19:38.386564+0000              0'0
>> 2022-06-03T08:33:01.050897+0000              0                    1
>> periodic scrub scheduled @ 2022-06-06T23:02:03.102683+0000
>>       0                0
>>
>> 3  22  0  0  0  0      5742  0  0     83     83
>> 1   5  0  0  0  0  14156320  0  0  10005  10005
>> 2   0  0  0  0  0         0  0  0      0      0
>>
>> sum  27  0  0  0  0  14162062  0  0  10088  10088
>> OSD_STAT  USED    AVAIL    USED_RAW  TOTAL    HB_PEERS
>>                                                                       
>>                 PG_SUM  PRIMARY_PG_SUM
>> 167       80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [0,11,34,35,90,91,92,93,94,95,96,97,98,133,166]       2
>>  166       80 GiB   16 TiB    80 GiB   16 TiB
>>                                     
>> [9,32,33,80,81,82,83,84,85,87,88,111,112,165,167]       2
>>    0
>> 77        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [3,22,23,76,78,101,109,154,157,158,159,160,161,162,163]       1
>>          0
>> 76        80 GiB   16 TiB    80 GiB   16 TiB
>>                        
>> [2,20,21,75,77,96,98,128,143,145,146,147,148,149,150,153,156]
>> 3               1
>> 75        80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [11,18,19,74,76,86,87,132,134,135,136,137,138,139,141]       1
>>         0
>> 74        80 GiB   16 TiB    80 GiB   16 TiB
>>      
>> [3,10,16,17,20,36,39,61,67,73,75,77,98,101,121,123,124,125,128,129,131,133,158]
>> 5
>> 2
>> 73        80 GiB   16 TiB    80 GiB   16 TiB
>>                    
>> [1,14,15,54,55,72,74,111,112,114,115,117,119,120,122,147,150,159]
>>    4               2
>> 72        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [0,12,13,40,41,71,73,100,101,103,106,107,108,109,110]       1
>>        0
>> 71        80 GiB   16 TiB    80 GiB   16 TiB
>>              
>> [8,24,34,35,48,70,72,82,89,90,91,94,95,96,97,98,99,144,150,157,158,161]
>>  3               1
>> 70        80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [9,32,33,69,71,78,79,80,84,85,86,87,88,148,150]       1
>>  69        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [7,30,31,65,68,70,71,72,73,74,75,76,77,132,136]       1
>>  68        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [6,28,29,53,58,60,61,62,63,64,66,67,69,128,130]       1
>>  67        80 GiB   16 TiB    80 GiB   16 TiB
>> [4,5,14,17,26,27,28,29,38,46,47,48,49,50,51,54,55,61,66,68,69,74,81,86,91,111,117,122,130]
>> 4
>> 2
>> 66        80 GiB   16 TiB    80 GiB   16 TiB
>>                                     
>> [4,24,25,36,37,40,41,42,45,46,65,67,101,108,165]       2
>>   0
>> 65        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [3,22,23,64,66,97,99,154,157,158,159,160,162,163,164]       2
>>        0
>> 64        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [2,20,21,63,65,78,88,143,146,147,148,149,150,151,155]       1
>>        0
>> 63        80 GiB   16 TiB    80 GiB   16 TiB
>>           
>> [11,18,19,28,31,47,62,64,72,74,80,104,107,132,134,136,137,138,139,140,144]
>>     3               1
>> 62        80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [10,16,17,49,50,61,63,121,123,125,127,128,129,130,133]       0
>>         0
>> 61        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [1,14,15,42,46,60,62,111,112,115,116,117,119,120,122]       2
>>        0
>> 60        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [0,12,13,59,61,100,101,102,105,107,108,109,110,159,160]       3
>>          0
>> 59        80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [8,34,35,45,58,60,89,90,91,95,96,97,98,99,129,147,150]       1
>>         1
>> 58        80 GiB   16 TiB    80 GiB   16 TiB
>>              
>> [9,32,33,45,49,50,57,59,78,79,80,81,84,85,86,87,88,112,129,137,140,148]
>>  1               1
>> 57        80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [7,30,31,56,58,65,69,70,72,73,74,76,77,121,129]       1
>>  56        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [6,28,29,53,55,57,58,59,60,61,63,64,66,113,119]       0
>>  55        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [5,26,27,38,47,48,49,50,51,52,54,56,57,100,107]       1
>>  54        80 GiB   16 TiB    80 GiB   16 TiB
>>         
>> [4,15,20,24,25,33,36,37,39,40,41,44,46,53,55,76,88,89,91,102,142,146,150,165]
>> 3
>> 2
>> 53        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [3,22,23,52,54,86,87,154,157,158,159,160,162,163,164]       1
>>        0
>> 52        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [2,20,21,51,53,65,70,143,146,148,149,150,151,152,153]       2
>>        0
>> 51        80 GiB   16 TiB    80 GiB   16 TiB
>> [0,1,3,11,18,19,34,38,41,42,50,52,60,61,66,80,83,86,96,99,132,134,135,137,138,140,141,142,143,144,145,164]
>> 3
>> 3
>> 50        80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [10,16,17,37,46,49,51,121,123,124,127,128,129,130,133]       2
>>         0
>> 49        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [1,14,15,48,50,111,112,113,114,115,116,119,122,160,163]       3
>>          0
>> 48        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [0,12,13,47,49,100,101,102,103,104,107,108,109,148,152]       2
>>          0
>> 47        80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [8,34,35,46,48,89,90,91,95,96,97,98,99,136,138]       1
>>  46        80 GiB   16 TiB    80 GiB   16 TiB
>>                   
>> [9,28,32,33,37,45,47,74,78,79,80,83,84,85,86,88,91,127,128,137,159]
>>      2               1
>> 45        80 GiB   16 TiB    80 GiB   16 TiB
>>             
>> [7,13,30,31,44,46,61,63,65,68,69,71,72,73,76,77,101,104,114,119,143,157]
>>   4               1
>> 44        80 GiB   16 TiB    80 GiB   16 TiB
>>               
>> [6,28,29,43,45,53,58,59,60,61,62,63,66,71,80,81,86,94,100,107,132,166]
>> 2               1
>> 43        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [5,26,27,38,42,44,47,48,49,50,51,52,54,91,96]       2               0
>> 42        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [4,24,25,36,37,39,40,41,43,44,45,46,79,83,167]       1               0
>> 41        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [3,22,23,40,42,72,73,154,156,158,159,161,162,164,166]       3
>>        0
>> 40        80 GiB   16 TiB    80 GiB   16 TiB
>>   
>> [2,10,11,18,20,21,39,41,43,58,66,76,96,99,128,135,143,145,146,147,148,151,152,153]
>> 6
>> 2
>> 39        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [11,18,19,38,40,50,132,135,136,137,138,140,141,142,167]       1
>>          0
>> 38        80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [10,16,17,37,39,121,124,125,126,127,128,129,130,164,166]       2
>>           0
>> 37        80 GiB   16 TiB    80 GiB   16 TiB
>>                        
>> [1,14,15,26,36,38,97,111,113,114,116,117,118,119,122,145,148]
>> 4               1
>> 36        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [0,12,19,35,37,100,101,102,103,104,105,107,108,109,141]       2
>>          0
>> 15        80 GiB   16 TiB    80 GiB   16 TiB
>>            
>> [4,8,10,11,14,16,26,73,81,96,132,134,135,136,137,138,139,140,141,142,144]
>>    2               1
>> 14        80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [10,13,15,25,121,123,124,125,126,127,128,129,130,131,133]       2
>>            0
>> 13        80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [1,12,14,22,111,112,113,114,115,116,117,118,119,120,122]       5
>>           0
>> 12        80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [1,11,13,17,100,101,102,103,104,105,106,107,108,109,110]       0
>>           0
>> 11        80 GiB   16 TiB    80 GiB   16 TiB
>>               
>> [7,10,12,14,21,35,49,89,90,91,92,93,94,95,96,97,98,99,104,132,134,149]
>> 3               1
>> 10        80 GiB   16 TiB    80 GiB   16 TiB
>>                                 
>> [9,11,13,32,33,78,79,80,81,82,83,84,85,86,87,88,131]       4
>>       1
>> 9         80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [8,10,30,31,65,68,69,70,71,72,73,74,75,76,77]       2               0
>> 8         80 GiB   16 TiB    80 GiB   16 TiB
>>                          
>> [3,7,9,13,28,29,31,44,53,56,58,59,60,61,62,63,64,66,67,132]       4
>>              1
>> 7         80 GiB   16 TiB    80 GiB   16 TiB
>>                                         
>> [6,8,26,27,38,43,47,48,49,50,51,52,54,55,57]       1               0
>> 6         80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [5,7,24,25,36,37,39,40,41,42,44,45,46,165,167]       0               0
>> 1         80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [0,2,14,15,111,112,113,114,115,116,117,118,119,120,122]       1
>>          0
>> 0         80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [1,12,13,100,101,102,103,104,105,106,107,108,109,110,167]       2
>>            0
>> 2         80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [1,3,16,17,121,123,124,125,126,127,128,129,130,131,133]       1
>>          0
>> 3         80 GiB   16 TiB    80 GiB   16 TiB
>>                      
>> [2,4,18,19,110,132,134,135,136,137,138,139,140,141,142,144,166]
>>  5               1
>> 4         80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [3,5,20,21,143,145,146,147,148,149,150,151,152,153,155]       2
>>          0
>> 5         80 GiB   16 TiB    80 GiB   16 TiB
>>         
>> [0,2,4,6,22,23,37,41,91,107,118,154,156,157,158,159,160,161,162,163,164,166]
>>       1               1
>> 16        80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [2,15,17,29,143,145,146,147,148,149,150,151,152,153,155]       0
>>           0
>> 17        80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [3,16,18,30,154,156,157,158,159,160,161,162,163,164,166]       3
>>           0
>> 18        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [4,17,19,33,36,37,39,40,41,42,44,45,46,131,148,165,167]       3
>>          1
>> 19        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [5,18,20,34,38,43,47,48,49,50,51,52,54,55,57]       2               0
>> 20        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [6,13,19,21,53,56,58,59,60,61,62,63,64,66,67]       4               0
>> 21        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [7,14,20,22,65,68,69,70,71,72,73,74,75,76,77]       3               0
>> 22        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [9,17,21,23,78,79,80,81,82,83,84,85,86,87,88]       0               0
>> 23        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [8,18,22,24,89,90,91,92,93,94,95,96,97,98,99]       1               0
>> 24        80 GiB   16 TiB    80 GiB   16 TiB
>>                        
>> [0,8,21,23,25,52,100,101,102,103,104,105,106,107,108,109,110]
>> 3               1
>> 25        80 GiB   16 TiB    80 GiB   16 TiB
>>                        
>> [1,9,22,24,26,93,111,112,113,114,115,116,117,118,119,120,122]
>> 3               1
>> 26        80 GiB   16 TiB    80 GiB   16 TiB
>>                    
>> [10,25,27,29,105,121,123,124,125,126,127,128,129,130,131,133,162]
>>    2               1
>> 27        80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [11,26,28,30,132,134,135,136,137,138,139,140,141,142,144]       1
>>            0
>> 28        80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [2,27,29,33,143,145,146,147,148,149,150,151,152,153,155]       4
>>           0
>> 29        80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [3,28,30,34,154,156,157,158,159,160,161,162,163,164,166]       1
>>           0
>> 30        80 GiB   16 TiB    80 GiB   16 TiB
>>                       
>> [4,13,29,31,36,37,39,40,41,42,44,45,46,65,109,133,136,165,167]
>> 3               1
>> 31        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [5,14,30,32,38,43,47,48,49,50,51,52,54,55,57]       4               0
>> 32        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [6,17,31,33,53,56,58,59,60,61,62,63,64,66,67]       1               0
>> 33        80 GiB   16 TiB    80 GiB   16 TiB
>>                                  
>> [7,18,30,32,34,41,65,68,69,70,71,72,73,74,75,76,77]       2
>>      1
>> 34        80 GiB   16 TiB    80 GiB   16 TiB
>>                                        
>> [9,21,33,35,78,79,80,81,82,83,84,85,86,87,88]       4               0
>> 35        80 GiB   16 TiB    80 GiB   16 TiB
>>       
>> [8,16,28,34,36,40,60,70,89,90,91,92,93,95,96,97,98,99,106,126,127,128,139,147]
>> 2
>> 2
>> 78        80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [4,24,25,36,39,40,41,42,44,45,46,77,79,111,120]       0
>>  79        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [5,26,27,47,48,49,50,51,52,54,57,78,80,123,124]       0
>>  80        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [6,28,29,53,58,59,61,63,64,66,67,79,81,132,136]       3
>>  81        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [7,30,31,65,68,71,73,74,75,76,77,80,82,146,148]       3
>>  82        80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [9,32,33,78,79,80,81,83,84,85,86,87,88,158,163]       2
>>  83        80 GiB   16 TiB    80 GiB   16 TiB
>>                
>> [8,21,32,34,35,36,40,72,82,84,89,90,91,92,96,97,98,99,114,121,122,153]
>> 4               2
>> 84        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [0,12,13,48,57,83,85,100,101,102,103,106,107,109,110]       1
>>        0
>> 85        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [1,14,15,61,66,84,86,111,112,113,114,115,116,117,120]       3
>>        0
>> 86        80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [10,16,17,74,75,85,87,121,123,124,125,126,128,129,131]       3
>>         0
>> 87        80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [11,18,19,86,88,98,99,132,134,135,136,137,138,139,140]       0
>>         0
>> 88        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [2,20,21,87,89,101,107,143,146,147,148,149,150,151,155]       2
>>          0
>> 89        80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [3,22,23,88,90,111,120,157,158,159,160,161,162,163,166]       0
>>          0
>> 90        80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [4,24,25,36,37,39,40,41,44,45,46,89,91,123,124]       1
>>  91        80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [5,26,27,38,47,48,49,50,52,54,57,90,92,120,132,136,142]       5
>>          1
>> 92        80 GiB   16 TiB    80 GiB   16 TiB
>>                 
>> [6,24,25,28,29,53,56,59,61,63,64,66,67,85,91,93,101,113,145,146,150]
>>       2               1
>> 93        80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [7,30,31,65,68,72,73,74,75,76,77,92,94,158,159]       3
>>  94        80 GiB   16 TiB    80 GiB   16 TiB
>>                                         
>> [9,32,33,40,41,78,79,80,81,85,86,87,88,93,95]       1               0
>> 95        80 GiB   16 TiB    80 GiB   16 TiB
>>                                  
>> [8,31,34,35,47,55,57,89,90,91,92,93,94,96,97,98,99]       1
>>      1
>> 96        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [0,12,13,61,64,95,97,100,101,102,103,105,106,109,110]       3
>>        0
>> 97        80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [1,14,15,75,77,96,98,111,112,113,114,115,117,118,122]       2
>>        0
>> 98        80 GiB   16 TiB    80 GiB   16 TiB
>>                        
>> [10,16,17,66,86,87,97,99,121,123,124,125,126,128,129,130,132]
>> 3               1
>> 99        80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [11,18,19,98,100,110,111,134,135,136,137,138,139,140,142]       3
>>            0
>> 100       80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [2,20,21,99,101,122,123,146,147,148,149,150,151,152,153]       0
>>           0
>> 101       80 GiB   16 TiB    80 GiB   16 TiB
>>                   
>> [3,20,22,23,33,74,100,102,132,135,158,159,160,161,162,163,164,166]
>>     3               0
>> 102       80 GiB   16 TiB    80 GiB   16 TiB
>>                                   
>> [4,24,25,36,37,39,40,41,44,45,101,103,145,147,165]       1
>>     0
>> 103       80 GiB   16 TiB    80 GiB   16 TiB
>>                                    
>> [5,26,27,38,43,47,48,49,52,54,57,102,104,157,160]       1
>>    0
>> 104       80 GiB   16 TiB    80 GiB   16 TiB
>>               
>> [6,17,28,29,40,41,46,53,56,59,62,63,64,66,67,73,82,84,103,105,110,147]
>> 5               1
>> 105       80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [7,30,31,47,50,65,68,70,72,74,75,76,77,104,106]       2
>>  106       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [9,32,33,58,64,78,79,80,81,84,85,86,88,105,107]       2
>>  107       80 GiB   16 TiB    80 GiB   16 TiB
>> [8,17,20,25,27,34,35,69,75,83,89,90,91,92,93,95,97,98,99,105,106,108,113,122,124,129,132,154]
>> 4
>> 2
>> 108       80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [0,12,13,86,87,100,101,102,103,104,105,106,107,109,110]       1
>>          0
>> 109       80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [1,14,15,96,98,108,110,111,112,113,114,115,116,117,118]       2
>>          0
>> 110       80 GiB   16 TiB    80 GiB   16 TiB
>>                           
>> [10,16,17,100,109,111,122,123,124,125,126,127,128,129,130]       4
>>             0
>> 111       80 GiB   16 TiB    80 GiB   16 TiB
>>                           
>> [11,18,19,110,112,123,131,135,136,137,138,139,140,142,144]       0
>>             0
>> 112       80 GiB   16 TiB    80 GiB   16 TiB
>>                        
>> [2,20,21,111,113,132,135,142,147,148,149,150,151,152,153,155]
>> 4               1
>> 113       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [3,22,23,112,114,145,147,154,159,160,161,162,163,164,166]       2
>>            0
>> 114       80 GiB   16 TiB    80 GiB   16 TiB
>>                                  
>> [4,24,25,37,39,40,41,44,45,113,115,157,159,165,167]       2
>>      0
>> 115       80 GiB   16 TiB    80 GiB   16 TiB
>>                                   
>> [5,26,27,38,43,45,46,48,49,50,52,55,57,68,114,116]       1
>>     1
>> 116       80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [6,28,29,47,48,53,56,60,61,62,63,66,67,115,117]       1
>>  117       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [7,30,31,61,64,65,68,69,70,72,75,76,77,116,118]       1
>>  118       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [9,32,33,71,75,78,79,80,81,82,86,87,88,117,119]       2
>>  119       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [8,34,35,84,88,89,90,91,92,93,94,97,99,118,120]       1
>>  120       80 GiB   16 TiB    80 GiB   16 TiB
>>                               
>> [0,12,13,90,96,101,102,103,104,105,106,107,109,119,121]       1
>>          0
>> 121       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [1,14,15,109,110,112,113,114,115,116,117,118,119,120,122]       2
>>            0
>> 122       80 GiB   16 TiB    80 GiB   16 TiB
>>                    
>> [10,16,17,71,111,112,121,123,124,125,126,127,128,129,130,131,132]
>>    3               1
>> 123       80 GiB   16 TiB    80 GiB   16 TiB
>>                           
>> [11,18,19,122,124,132,135,136,137,138,139,140,141,142,144]       0
>>             0
>> 124       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [2,20,21,123,125,143,145,147,148,149,150,151,152,153,155]       1
>>            0
>> 125       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [3,22,23,124,126,154,156,157,159,160,161,162,163,164,166]       0
>>            0
>> 126       80 GiB   16 TiB    80 GiB   16 TiB
>>                                    
>> [4,24,25,36,37,39,41,42,44,45,46,125,127,165,167]       1
>>    0
>> 127       80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [5,26,27,38,43,47,48,49,50,51,52,55,57,126,128]       2
>>  128       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [6,28,29,53,56,58,60,61,62,63,64,66,67,127,129]       2
>>  129       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [7,30,31,65,68,69,70,72,73,74,75,76,77,128,130]       3
>>  130       80 GiB   16 TiB    80 GiB   16 TiB
>>        
>> [9,32,33,43,45,65,66,78,79,80,81,82,83,84,86,87,88,93,103,129,131,143,145,167]
>> 3
>> 2
>> 131       80 GiB   16 TiB    80 GiB   16 TiB
>>                
>> [8,13,34,35,63,89,90,91,92,93,94,95,96,97,99,130,132,133,135,146,151]
>> 4               1
>> 132       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [0,12,13,101,102,103,104,105,106,107,108,109,110,131,133]       7
>>            0
>> 133       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [1,14,15,111,112,113,114,115,116,117,118,119,120,132,134]       2
>>            0
>> 134       80 GiB   16 TiB    80 GiB   16 TiB
>>                           
>> [10,16,17,121,124,125,126,127,128,129,130,131,133,135,144]       3
>>             0
>> 135       80 GiB   16 TiB    80 GiB   16 TiB
>> [8,11,18,19,88,98,112,114,119,131,132,134,136,137,138,139,140,141,142,144,145,147,154]
>> 4
>> 2
>> 136       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [2,20,21,135,137,143,145,149,150,151,152,153,155,157,159]       1
>>            0
>> 137       80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [3,22,23,41,46,136,138,154,156,157,160,161,162,164,166]       1
>>          0
>> 138       80 GiB   16 TiB    80 GiB   16 TiB
>>                                    
>> [4,24,25,37,39,42,44,45,46,47,50,137,139,165,167]       0
>>    0
>> 139       80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [5,26,27,38,43,47,49,51,52,54,55,58,67,138,140]       1
>>  140       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [6,28,29,53,56,58,59,60,62,64,66,73,77,139,141]       0
>>  141       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [7,30,31,65,68,69,70,71,72,73,76,85,87,140,142]       1
>>  142       80 GiB   16 TiB    80 GiB   16 TiB
>>                                       
>> [9,32,33,78,80,81,82,83,84,85,86,90,98,141,143]       3
>>  143       80 GiB   16 TiB    80 GiB   16 TiB
>>                                     
>> [8,34,35,89,91,92,93,94,95,96,98,107,109,142,144]       4
>>    0
>> 144       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [0,12,13,102,103,104,105,106,107,108,110,111,112,143,145]       3
>>            0
>> 145       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [1,14,15,111,114,115,116,117,118,119,120,124,133,144,146]       3
>>            0
>> 146       80 GiB   16 TiB    80 GiB   16 TiB
>> [10,16,17,104,106,108,121,126,127,128,129,130,131,133,134,136,141,145,147,158,159,163]
>> 3
>> 1
>> 147       80 GiB   16 TiB    80 GiB   16 TiB
>>                    
>> [11,18,19,34,132,134,138,139,140,141,142,144,146,148,157,159,167]
>>    5               1
>> 148       80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [2,20,21,40,46,143,145,146,147,149,150,151,152,153,155]       2
>>          0
>> 149       80 GiB   16 TiB    80 GiB   16 TiB
>>       
>> [3,11,14,19,22,23,47,48,60,85,110,143,148,150,154,156,157,158,161,162,164,166]
>> 3
>> 2
>> 150       80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [4,20,21,24,25,36,37,39,42,45,46,61,67,149,151,165,167]       5
>>          2
>> 151       80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [5,26,27,38,43,47,50,51,52,54,55,71,77,150,152]       2
>>  152       80 GiB   16 TiB    80 GiB   16 TiB
>>                                  
>> [3,6,28,29,53,56,58,59,60,61,62,66,85,87,93,151,153]       2
>>       1
>> 153       80 GiB   16 TiB    80 GiB   16 TiB
>>                                      
>> [7,30,31,65,69,70,71,72,73,75,76,90,98,152,154]       1
>>  154       80 GiB   16 TiB    80 GiB   16 TiB
>>                                     
>> [9,32,33,78,80,81,82,83,84,85,88,107,109,153,155]       2
>>    0
>> 155       80 GiB   16 TiB    80 GiB   16 TiB
>>                                    
>> [8,34,35,91,92,93,94,95,96,97,98,111,112,154,156]       0
>>    0
>> 156       80 GiB   16 TiB    80 GiB   16 TiB
>>                     
>> [0,12,13,40,100,103,104,105,106,107,108,109,110,124,133,155,157]
>>   2               1
>> 157       80 GiB   16 TiB    80 GiB   16 TiB
>>                            
>> [1,14,15,111,115,116,117,118,119,120,122,134,136,156,158]       1
>>            0
>> 158       80 GiB   16 TiB    80 GiB   16 TiB
>>                           
>> [10,16,17,121,123,126,127,129,130,131,133,146,148,157,159]       3
>>             0
>> 159       80 GiB   16 TiB    80 GiB   16 TiB
>>                             
>> [11,18,19,40,46,132,134,135,138,140,141,142,144,158,160]       3
>>           0
>> 160       80 GiB   16 TiB    80 GiB   16 TiB
>>                       
>> [2,20,21,47,49,54,112,143,145,146,147,150,151,152,155,159,161]
>> 2               1
>> 161       80 GiB   16 TiB    80 GiB   16 TiB
>>                              
>> [3,22,23,61,67,154,156,157,158,159,160,162,163,164,166]       1
>>          0
>> 162       80 GiB   16 TiB    80 GiB   16 TiB
>>                
>> [4,9,10,23,24,25,36,37,39,40,42,46,71,74,121,152,160,161,163,165,167]
>> 2               1
>> 163       80 GiB   16 TiB    80 GiB   16 TiB
>>            
>> [5,26,27,30,38,43,47,50,51,52,54,55,57,85,88,110,116,118,147,151,162,164]
>>    2               1
>> 164       80 GiB   16 TiB    80 GiB   16 TiB
>>                                
>> [6,28,29,31,34,53,59,60,61,62,64,66,67,90,93,163,165]       2
>>        1
>> 165       80 GiB   16 TiB    80 GiB   16 TiB
>>                                    
>> [7,30,31,65,69,70,71,72,73,74,75,109,110,164,166]       0
>>    0
>> sum       13 TiB  2.7 PiB    13 TiB  2.7 PiB
>>
>> * NOTE: Omap statistics are gathered during deep scrub and may be
>> inaccurate soon afterwards depending on utilization. See
>> http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics
>> for further details.
>> dumped all
>>
>>
>> root@store-par2-node01:/home/user# ceph fs status
>> cephfs - 2 clients
>> ======
>> RANK  STATE                 MDS                   ACTIVITY     DNS
>>  INOS   DIRS   CAPS
>>  0    active  cephfs.store-par2-node01.honsru  Reqs:    0 /s    10
>>   13     12      2
>>       POOL         TYPE     USED  AVAIL
>> cephfs_metadata  metadata  5742    870T
>>   cephfs_data      data       0   1959T
>>           STANDBY MDS
>> cephfs.store-par2-node02.ttsfvq
>> cephfs.store-par2-node03.vnxscc
>> cephfs.store-par2-node04.iudcse
>>
>>
>> If somebody have an idea how can I correct this, or good pointers to
>> learn about PG with cephadm, It will be nice ...
>>
>> Thanks
>>
>>
>> --
>> Christophe BAILLON
>> Mobile :: +336 16 400 522
>> Work :: https://eyona.com
>> Twitter :: https://twitter.com/ctof
>>
>>
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx
>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
> 
> 
> 
> 
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx

-- 
Christophe BAILLON
Mobile :: +336 16 400 522
Work :: https://eyona.com
Twitter :: https://twitter.com/ctof


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux