On 08/19/2015 07:36 AM, Dałek, Piotr wrote:
-----Original Message-----
From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel-
owner@xxxxxxxxxxxxxxx] On Behalf Of Mark Nelson
Sent: Wednesday, August 19, 2015 2:17 PM
The RSS memory usage in the report is per OSD I guess(really?). It
can't be ignored since it's really a great improvement memory usage.
Do you mean with tcmalloc? I think it's a tough decision. For jemalloc, 300MB
more of RSS per OSD does add up (about 18GB for 60 OSDs). On the other
hand, the cost of memory is such a small fraction of the overall cost of
systems like this that it might be worth it to switch over anyway. In the 4K
write tests it's pretty clear that even with 128MB TC, TCMalloc is suffering
and jemalloc appears to still have headroom left. It's possible that bumping
the thread cache even higher might help TCMalloc close the gap though. It's
also possible that jemalloc might have worse memory behavior under
recovery scenarios as we discussed at the hackathon (And Somnath
mentioned above), so I think we probably need to run the tests.
Have you tried running these tests again with TCMalloc after applying patches from https://github.com/ceph/ceph/pull/5534? Because switching to jemalloc alone won't fix the root issues, only make it less painful.
Not yet, my first goal was to simply replicate the tests that have
already been done using hammer to get a baseline on the new cluster. It
definitely looks like that's (yet another!) good next step.
Mark
With best regards / Pozdrawiam
Piotr Dałek
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
