Re: osd_memory_target for low-memory machines

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



There is definitely something wrong in how my cluster manages osd_memory_target. For example, this is the situation for OSD 16:

# ceph config get osd.6 osd_memory_target
280584038
# ceph orch ps ogion --daemon_type osd --daemon_id 16
NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID osd.16 ogion running (21m) 6m ago 3d 568M 384M 17.2.3 df36836ad61e 35bf9c20dbeb

The memory limit seems to be correctly set (I disabled the memory autotune on the host, set the limit manually with --force and rebooted the host) but nevertheless the actual memory usage is far above the limit (568M vs 384M).

How can this be? Am I overlooking something?

On 02/10/22 19:16, Nicola Mori wrote:
I attach two files with the requested info. Some more details: the cluster has been deployed with cephadm using Ceph 17.2.3 image tweaked with a small addition (a custom module for alerts). I guess the configuration is almost vanilla since I changed very few things.
Thanks for your help.

Nicola

On 02/10/22 17:21, Joseph Mundackal wrote:
can you share `ceph daemon osd.8 config show` and `ceph config dump`?


On Sun, Oct 2, 2022 at 5:10 AM Nicola Mori <mori@xxxxxxxxxx <mailto:mori@xxxxxxxxxx>> wrote:

    Dear Ceph users,

    I put together a cluster by reusing some (very) old machines with low
    amounts of RAM, as low as 4 GB for the worst case. I'd need to set
    osd_memory_target properly to avoid going OOM, but it seems there is a
    lower limit preventing me to do so consistently:

    1) in the cluster logs I continuously get this message:

        Unable to set osd_memory_target on ceph01 to 402653184: error
    parsing
    value: Value '831882342' is below minimum 939524096

    2) if I try to set it manually I get this error:

        # ceph config set osd.7 osd_memory_target 402653184
        Error EINVAL: error parsing value: Value '402653184' is below
    minimum
    939524096

    3) if I use the --force option then the value is set, but it is lost
    after a osd restart

    How can I fix this issue?
    Thanks,

    Nicola
    _______________________________________________
    ceph-users mailing list -- ceph-users@xxxxxxx
    <mailto:ceph-users@xxxxxxx>
    To unsubscribe send an email to ceph-users-leave@xxxxxxx
    <mailto:ceph-users-leave@xxxxxxx>



--
Nicola Mori, Ph.D.
INFN sezione di Firenze
Via Bruno Rossi 1, 50019 Sesto F.no (Italy)
+390554572660
mori@xxxxxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux