Yes, Slab containers helps here by amortizing the malloc overhead for small nodes. Allen Samuels SanDisk |a Western Digital brand 951 SanDisk Drive, Milpitas, CA 95035 T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@xxxxxxxxxxx > -----Original Message----- > From: Igor Fedotov [mailto:ifedotov@xxxxxxxxxxxx] > Sent: Tuesday, December 20, 2016 9:08 AM > To: Sage Weil <sage@xxxxxxxxxxxx> > Cc: Allen Samuels <Allen.Samuels@xxxxxxxxxxx>; ceph-devel <ceph- > devel@xxxxxxxxxxxxxxx> > Subject: Re: Bug in mempool::map? > > Some update on map<uint64_t, uint32_t> mem usage. > > It looks like single entry map takes 48 bytes. And 40 bytes for > map<uint32_t,uint32_t>. > > Hence 1024 trivial ref_maps for 1024 blobs takes >48K! > > These are my results taken from mempools. And they look pretty similar > to what's been said in the following article: > > http://lemire.me/blog/2016/09/15/the-memory-usage-of-stl-containers- > can-be-surprising/ > > > Sage, you mentioned that you're planning to do something with ref maps > during the standup but I missed the details. Is that something about > their mem use or anything else? > > > Thanks, > > Igor > > > > On 20.12.2016 18:25, Sage Weil wrote: > > On Tue, 20 Dec 2016, Igor Fedotov wrote: > >> Hi Allen, > >> > >> It looks like mempools don't measure maps allocations properly. > >> > >> I extended unittest_mempool in the following way but corresponding > output is > >> always 0 for both 'before' and 'after' values: > >> > >> > >> diff --git a/src/test/test_mempool.cc b/src/test/test_mempool.cc > >> index 4113c53..b38a356 100644 > >> --- a/src/test/test_mempool.cc > >> +++ b/src/test/test_mempool.cc > >> @@ -232,9 +232,19 @@ TEST(mempool, set) > >> TEST(mempool, map) > >> { > >> { > >> - mempool::unittest_1::map<int,int> v; > >> - v[1] = 2; > >> - v[3] = 4; > >> + size_t before = mempool::buffer_data::allocated_bytes(); > > I think it's just that you're measuring the buffer_data pool... > > > >> + mempool::unittest_1::map<int,int>* v = new > mempool::unittest_1::map<int,int>; > > but the map is in the unittest_1 pool? > > > >> + (*v)[1] = 2; > >> + (*v)[3] = 4; > >> + size_t after = mempool::buffer_data::allocated_bytes(); > >> + cout << "before " << before << " after " << after << std::endl; > >> + delete v; > >> + before = after; > >> + mempool::unittest_1::map<int64_t,int64_t> v2; > >> + v2[1] = 2; > >> + v2[3] = 4; > >> + after = mempool::buffer_data::allocated_bytes(); > >> + cout << " before " << before << " after " << after << std::endl; > >> } > >> { > >> mempool::unittest_2::map<int,obj> v; > >> > >> > >> Output: > >> > >> [ RUN ] mempool.map > >> before 0 after 0 > >> before 0 after 0 > >> [ OK ] mempool.map (0 ms) > >> > >> It looks like we do not measure ref_map for BlueStore Blob and > SharedBlob > >> classes too. > >> > >> Any ideas? > >> > >> Thanks, > >> > >> Igor > >> > >> -- > >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >> the body of a message to majordomo@xxxxxxxxxxxxxxx > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> > >> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
