On 5/3/19 1:38 AM, Denny Fuchs wrote:
hi, I never recognized the Debian /etc/default/ceph :-) ========= # Increase tcmalloc cache size TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728 ======== that is, what is active now.
Yep, if you profile the OSD under a small write workload you can see how changing this affects tcmalloc's behavior. This was especially important back before we had the async messenger, but I believe we've seen evidence that we're still getting some benefit with larger thread cache even now.
Huge pages: # cat /sys/kernel/mm/transparent_hugepage/enabled always [madvise] never # dpkg -S /usr/lib/x86_64-linux-gnu/libjemalloc.so.1 libjemalloc1: /usr/lib/x86_64-linux-gnu/libjemalloc.so.1 so file exists on PROXMOX 5.x (Ceph version 12.2.11-pve1) If I understand correct: I should try to set bitmap allocator [osd] ... bluestore_allocator = bitmap bluefs_allocator = bitmap I would restart the nodes one by one and see, what happens.
If you are using 12.2.11 you likely still have the old bitmap allocator which we do not recommend using at all. Igor Fedotov backported his excellent new bitmap allocator to 12.2.12 though:
https://github.com/ceph/ceph/tree/v12.2.12/src/os/bluestore Mark
cu denny _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
