On Sat, Jun 6, 2015 at 2:07 PM, Dałek, Piotr <Piotr.Dalek@xxxxxxxxxxxxxx> wrote: >> -----Original Message----- >> From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel- >> >> I'm digging into perf and the code to see here/how I might be able to >> improve performance for small I/O around 16K. >> >> I ran fio with rados and used perf to record. Looking through the report, >> there is a very substantial amount of time creating threads (or so it looks, but >> I'm really new to perf). It seems to point to messenger, so I looked in the >> code. From perf if looks like thread pooling isn't happening, but from what I >> can gather from the code, it should. >> [..] > > This is so because you use SimpleMessenger, which can't handle small I/O well. > Indeed, threads are problematic with it, as well as memory allocation. I did some > benchmarking some time ago and the gist of it is that you could try going for > AsyncMessenger and see if it helps. You can also see my results here: > http://stuff.predictor.org.pl/chunksize.xlsx > From there you can see that most of the time of small I/Os in SimpleMessenger > Is spent in tcmalloc code, and also there's a performance drop around 64k > Blocksize in Async Messenger. > Thanks for your benchmark, I submit new performance enhanced patchset for AsyncMessenger. It should solve original stress test performance degraded problem :-) > With best regards / Pozdrawiam > Piotr Dałek > -- Best Regards, Wheat -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
