On Wed, 3 Dec 2014, Chaitanya Huilgol wrote: > (2) TCmalloc - Client -2 (note significant increase in TCmalloc internal free to central list code paths) > > 14.75% libtcmalloc.so.4.1.2 [.] tcmalloc::CentralFreeList::FetchFromSpans() > 7.46% libtcmalloc.so.4.1.2 [.] tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int) > 6.71% libtcmalloc.so.4.1.2 [.] tcmalloc::CentralFreeList::ReleaseToSpans(void*) > 1.68% libtcmalloc.so.4.1.2 [.] operator new(unsigned long) > 1.57% ceph-osd [.] crush_hash32_3 Yikes! > IMHO, we should probably look at the following in general for better > performance with less variation > > - Add jemalloc option for ceph builds Definitely. Several years ago we saw serious heap fragmetnation issues with glibc. I suspect newer versions are less problematics. It may also be that newer version of tcmalloc behave better (not sure if we're linking against the latest version?). In any case, we should have build support for all options. We'll need to be careful when making a change, though. The best choice may also vary on a per-distro basis. > - Look at ways to evenly distribute PGs across the shards - with larger > number of shards some shards do not get exercised at all while some are > overloaded Ok > - Look at decreasing heap activity in the I/O path (index Manager, Hash > Index, LFN index etc.) Yes. Unfortunately I think this is a long tail... lots of small changes needed before we'll see much impact. sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
