On Sat, 8 Jun 2019 at 04:35, Sergei Genchev <sgenchev@xxxxxxxxx> wrote: > > Hi, > My OSD processes are constantly getting killed by OOM killer. My > cluster has 5 servers, each with 18 spinning disks, running 18 OSD > daemons in 48GB of memory. > I was trying to limit OSD cache, according to > http://docs.ceph.com/docs/mimic/rados/configuration/bluestore-config-ref/ > > [osd] > bluestore_cache_size_ssd = 1G > bluestore_cache_size_hdd = 768M > Yet, my OSDs are using way more memory than that. I have seen as high as 3.2G Well, it's widely known for a long time that 640KB isn't enough for everyone. ;) CEPH's OSD RAM consumption is largely dependent on its backing store capacity. Check out official recommendations: http://docs.ceph.com/docs/luminous/start/hardware-recommendations/ -- "ceph-osd: RAM~1GB for 1TB of storage per daemon". You didn't specify capacity of the disks, BTW. 2--3 TB? [...] > Is there any way for me to limit how much memory does OSD use? Try adding to same [osd] section osd_memory_target = ...Amount_in_Bytes... Don't set it to 640 KB though. ;-) minimum recommendations make sense still, reduce with caution. -- End of message. Next message? _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |