On Sun, Nov 17 2019 at 8:50pm -0500, Jeffle Xu <jefflexu@xxxxxxxxxxxxxxxxx> wrote: > Single thread fio test (read, bs=4k, ioengine=libaio, iodepth=128, > numjobs=1) over dm-thin device has poor performance versus bare nvme > disk. > > Further investigation with perf indicates that queue_work_on() consumes > over 20% CPU time when doing IO over dm-thin device. The call stack is > as follows. > > - 40.57% thin_map > + 22.07% queue_work_on > + 9.95% dm_thin_find_block > + 2.80% cell_defer_no_holder > 1.91% inc_all_io_entry.isra.33.part.34 > + 1.78% bio_detain.isra.35 > > In cell_defer_no_holder(), wakeup_worker() is always called, no matter > whether the tc->deferred_bio_list list is empty or not. In single thread > IO model, this list is most likely empty. So skip waking up worker thread > if tc->deferred_bio_list list is empty. > > A significant IO performance of single thread can be seen with this patch. > The original IO performance is 448 MiB/s with the fio test previously > described, while it is 646 MiB/s after applying the patch, which is a > 44% performance improvement. > > Signed-off-by: Jeffle Xu <jefflexu@xxxxxxxxxxxxxxxxx> > --- > Changes since V1: > 1. Check whether tc->deferred_bio_list list is empty or not, rather than > cell->bios as before. > 2. Retest the performance with the new version of this patch, and update > the statistics in the commit message. I've staged this with slight revisions to the header and renamed the empty variable to has_work, please see: https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-5.5&id=d256d796279de0bdc227ff4daef565aa7e80c898 Thanks. -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
