Hi,
> On 13 Aug 2021, at 14:10, Martin Traxl <martin.traxl@xxxxxxxx> wrote:
>
> yesterday evening one of my rgw nodes died again, radosgw was killed by the kernel oom killer.
>
> [Thu Aug 12 22:10:04 2021] Out of memory: Killed process 1376 (radosgw) total-vm:70747176kB, anon-rss:63900544kB, file-rss:0kB, shmem-rss:0kB, UID:167 pgtables:131008kB oom_score_adj:0
> [Thu Aug 12 22:10:09 2021] oom_reaper: reaped process 1376 (radosgw), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
>
> The radosgw was eating up all the 64GB system memory.
> A few hours before this happened, mempool dump showed a total usage of only 2.1 GB of ram, while in fact radosgw was using already 84.7% of 64GB.
>
> "total": {
> "items": 88757980,
> "bytes": 2147532284
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 1376 ceph 20 0 58.8g 52.7g 17824 S 48.2 84.7 20158:04 radosgw
>
>
> It seems the radowgw loses track of some memory, like there is a memory leak.
>
> Some additional information. I am running on CentOS 8.4, kernel 4.18. As already mentioned, Ceph 14.2.22. radosgw is the only notable service running on this machine.
> Any suggestions on this? Are there maybe any tuning settings? How could I debug this further?
Please show your "config diff" from admin socket
Couple of days ago I was upgraded our RGW's to 14.2.21 to 14.2.22 and don't see increase memory consumption
Thanks,
k
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
