I'm expecting the "in-object" capabilities of the slab_xxxxx containers to dramatically reduce malloc/free calls. Also having some visibility into where the memory is actually going will also spur a bit more development that will likely have more significant positive effects. Sent from my iPhone. Please excuse all typos and autocorrects. > On Sep 30, 2016, at 4:22 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > >> On Wed, 28 Sep 2016, Allen Samuels wrote: >> Boost::pool works very well when you're allocated "same" sized objects. >> That's not our situation, we're allocating lots of different sized >> objects -- some small, some large. The only way that Boost::pool >> supports that situation is to use the "ordered_free" operation to keep >> the freelist sorted (if you don't use it then you'll get fragmentation >> that'll prevent allocation of large objects -- even though there's >> plenty of free memory). The implementation of the sorted freelist is >> O(N). Which should work well for small pools, but that's the exact >> opposite of the desired use for Ceph, we're targeting large pools (think >> 1GB). >> >> I didn't word it very well, but my proposal doesn't actually change the >> underlying malloc/free algorithm, rather it's intended to put some >> statistics around memory usage so that we can self-trim our memory >> pools. > > We were doing some heap profiling yesterday and one interesting thing is > that the utilized heap reported by tcmalloc is about 1/2 the RSS. We > probably want to consider creating separate pools for the handful of > objects that are consuming the bulk of the heap. > > We did this a few years back in the MDS and IIRC it helped significantly > with memory utilization there. > > sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
