Hi Hervé,
On 01.06.21 13:15, Hervé Ballans wrote:
# ceph status
cluster:
id: 838506b7-e0c6-4022-9e17-2d1cf9458be6
health: HEALTH_ERR
1 filesystem is degraded
3 full osd(s)
1 pool(s) full
1 daemons have recently crashed
You have full OSDs and therefore a full pool. The "fullness" of a pool
is limited by the fullest OSD, i.e. a single full OSDs can block your
pool. Take a look at `ceph osd df` and you will notice a very
non-uniform osd usage (both with numbers of PG / size as well as usage %).
osd: 126 osds: 126 up (since 5m), 126 in (since 5M)
pgs: 1662 active+clean
The PG/osd ratio seems to be very low for me. The general recommendation
is 100 PG / osd post-replication (and power of 2 for each pool). In my
cluster I actually run with ~200 PG / osd for my SSD which contain the
cephfs metadata.
Thanks a lot if you have some ways for trying to solve this...
You have to get your OSDs to rebalance, which probably includes
increasing the number of PGs in some pools. Details depend on which Ceph
version you are running and your CRUSH rules (maybe your cephfs metadata
pool is residing only on NVMe?). Take a look at the balancer module [1]
and the autoscaler [2] (`ceph osd pool autoscale-status` is most
interesting).
Theoretically, you could (temporarilly!) increase the full_ratio.
However, this is a very dangerous operation which you should not do
unless you know *exactly* what you are doing.
Cheers & Best of luck
Sebastian
[1] https://docs.ceph.com/en/latest/rados/operations/balancer/
[2] https://docs.ceph.com/en/latest/rados/operations/placement-groups/
Replace latest in the URIs with your Ceph version string (i.e. octopus,
nautilus) for version specific documentation
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
