What are your full ratios? The defaults are:
"mon_osd_backfillfull_ratio": "0.900000",
"mon_osd_full_ratio": "0.950000",
You could temporarily increase the mon_osd_backfillfull_ratio a bit
and see if it resolves. But it's not recommended to get an OSD really
full, so be careful with that. Do you have the option to add more disks?
Zitat von Marco Pizzolo <marcopizzolo@xxxxxxxxx>:
Hello Everyone,
We seem to be having a problem on one of our ceph clusters post the OS
patch and reboot of one of the nodes. The three other nodes are showing
OSD fill rates of 77%-81%, but the 60 OSDs contained in the host that was
just rebooted are varying between 64% and 90% since the reboot occurred.
The three other nodes have not yet been patched or rebooted.
The result is:
health: HEALTH_WARN
15 nearfull osd(s)
7 pool(s) nearfull
Low space hindering backfill (add storage if this doesn't
resolve itself): 15 pgs backfill_toofull
Degraded data redundancy: 170940/1437684990 objects degraded
(0.012%), 4 pgs degraded, 4 pgs undersized
services:
mon: 3 daemons, quorum prdceph01,prdceph02,prdceph03 (age 6h)
mgr: prdceph01(active, since 5w), standbys: prdceph02, prdceph03,
prdceph04
mds: ArchiveRepository:1 {0=prdceph01=up:active} 3 up:standby
osd: 240 osds: 240 up (since 6h), 240 in (since 27h); 16 remapped pgs
task status:
scrub status:
mds.prdceph01: idle
data:
pools: 7 pools, 8384 pgs
objects: 479.23M objects, 557 TiB
usage: 1.7 PiB used, 454 TiB / 2.1 PiB avail
pgs: 170940/1437684990 objects degraded (0.012%)
4155186/1437684990 objects misplaced (0.289%)
8332 active+clean
36 active+clean+scrubbing+deep
11 active+remapped+backfill_toofull
2 active+undersized+degraded+remapped+backfill_toofull
2
active+forced_recovery+undersized+degraded+remapped+forced_backfill+backfill_toofull
1 active+remapped+backfilling
io:
client: 9.6 MiB/s rd, 820 KiB/s wr, 1.02k op/s rd, 189 op/s wr
recovery: 0 B/s, 25 keys/s, 10 objects/s
Any suggestions would be greatly appreciated, as currently it is not able
to complete the repair, nor will it backfill, even when attempting to force.
Many thanks in advance.
Marco
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
