Dear team, below is the output of ceph df command and the ceph version I am running ceph df --- RAW STORAGE --- CLASS SIZE AVAIL USED RAW USED %RAW USED hdd 433 TiB 282 TiB 151 TiB 151 TiB 34.82 TOTAL 433 TiB 282 TiB 151 TiB 151 TiB 34.82 --- POOLS --- POOL ID PGS STORED OBJECTS USED %USED MAX AVAIL device_health_metrics 1 1 1.1 MiB 3 3.2 MiB 0 73 TiB .rgw.root 2 32 3.7 KiB 8 96 KiB 0 73 TiB default.rgw.log 3 32 3.6 KiB 209 408 KiB 0 73 TiB default.rgw.control 4 32 0 B 8 0 B 0 73 TiB default.rgw.meta 5 32 382 B 2 24 KiB 0 73 TiB volumes 6 128 21 TiB 5.68M 62 TiB 22.09 73 TiB images 7 32 878 GiB 112.50k 2.6 TiB 1.17 73 TiB backups 8 32 0 B 0 0 B 0 73 TiB vms 9 32 881 GiB 174.30k 2.5 TiB 1.13 73 TiB testbench 10 32 0 B 0 0 B 0 73 TiB root@ceph-mon1:~# ceph --version ceph version 16.2.11 (3cf40e2dca667f68c6ce3ff5cd94f01e711af894) pacific (stable) root@ceph-mon1:~# please advise accordingly Michel On Mon, Jan 29, 2024 at 9:48 PM Frank Schilder <frans@xxxxxx> wrote: > You will have to look at the output of "ceph df" and make a decision to > balance "objects per PG" and "GB per PG". Increase he PG count for the > pools with the worst of these two numbers most such that it balances out as > much as possible. If you have pools that see significantly more user-IO > than others, prioritise these. > > You will have to find out for your specific cluster, we can only give > general guidelines. Make changes, run benchmarks, re-evaluate. Take the > time for it. The better you know your cluster and your users, the better > the end result will be. > > Best regards, > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Michel Niyoyita <micou12@xxxxxxxxx> > Sent: Monday, January 29, 2024 2:04 PM > To: Janne Johansson > Cc: Frank Schilder; E Taka; ceph-users > Subject: Re: Re: 6 pgs not deep-scrubbed in time > > This is how it is set , if you suggest to make some changes please advises. > > Thank you. > > > ceph osd pool ls detail > pool 1 'device_health_metrics' replicated size 3 min_size 2 crush_rule 0 > object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode on last_change 1407 > flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application > mgr_devicehealth > pool 2 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash > rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 1393 flags > hashpspool stripe_width 0 application rgw > pool 3 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 > object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change > 1394 flags hashpspool stripe_width 0 application rgw > pool 4 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 > object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change > 1395 flags hashpspool stripe_width 0 application rgw > pool 5 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 > object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change > 1396 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw > pool 6 'volumes' replicated size 3 min_size 2 crush_rule 0 object_hash > rjenkins pg_num 128 pgp_num 128 autoscale_mode on last_change 108802 lfor > 0/0/14812 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd > removed_snaps_queue > [22d7~3,11561~2,11571~1,11573~1c,11594~6,1159b~f,115b0~1,115b3~1,115c3~1,115f3~1,115f5~e,11613~6,1161f~c,11637~1b,11660~1,11663~2,11673~1,116d1~c,116f5~10,11721~c] > pool 7 'images' replicated size 3 min_size 2 crush_rule 0 object_hash > rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 94609 flags > hashpspool,selfmanaged_snaps stripe_width 0 application rbd > pool 8 'backups' replicated size 3 min_size 2 crush_rule 0 object_hash > rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 1399 flags > hashpspool stripe_width 0 application rbd > pool 9 'vms' replicated size 3 min_size 2 crush_rule 0 object_hash > rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 108783 lfor > 0/561/559 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd > removed_snaps_queue [3fa~1,3fc~3,400~1,402~1] > pool 10 'testbench' replicated size 3 min_size 2 crush_rule 0 object_hash > rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 20931 lfor > 0/20931/20929 flags hashpspool stripe_width 0 > > > On Mon, Jan 29, 2024 at 2:09 PM Michel Niyoyita <micou12@xxxxxxxxx<mailto: > micou12@xxxxxxxxx>> wrote: > Thank you Janne , > > no need of setting some flags like ceph osd set nodeep-scrub ??? > > Thank you > > On Mon, Jan 29, 2024 at 2:04 PM Janne Johansson <icepic.dz@xxxxxxxxx > <mailto:icepic.dz@xxxxxxxxx>> wrote: > Den mån 29 jan. 2024 kl 12:58 skrev Michel Niyoyita <micou12@xxxxxxxxx > <mailto:micou12@xxxxxxxxx>>: > > > > Thank you Frank , > > > > All disks are HDDs . Would like to know if I can increase the number of > PGs > > live in production without a negative impact to the cluster. if yes which > > commands to use . > > Yes. "ceph osd pool set <poolname> pg_num <number larger than before>" > where the number usually should be a power of two that leads to a > number of PGs per OSD between 100-200. > > -- > May the most significant bit of your life be positive. > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |