On Thu, Jan 27, 2022 at 05:21:43PM +0800, Xiaoguang Wang wrote: > For block devices that call blk_mq_complete_request() to end request > in process context, it's pointless to redirect the completion to be > done in block soft-irq, also blk_mq_raise_softirq() isnt't a very > light operation, which contains preempt and hard irq disable, wake > up ksoftirqd in non-interrupt context. > > I found this issue while I use tcm_loop and tcmu(backstore is file) > to evaluate performance, tcm_loop end request in workqueue. > Without this patch: libaio engine, direct io, randwrite, io size 128k > job0: (groupid=0, jobs=1): err= 0: pid=20876: Thu Jan 27 15:33:45 2022 > write: IOPS=15.7k, BW=1966MiB/s (2062MB/s)(115GiB/60001msec); 0 zone resets > slat (nsec): min=5675, max=83552, avg=8689.69, stdev=996.96 > clat (usec): min=231, max=99977, avg=498.89, stdev=501.69 > lat (usec): min=291, max=99986, avg=507.70, stdev=501.69 > > With this patch: > job0: (groupid=0, jobs=1): err= 0: pid=12813: Thu Jan 27 15:50:46 2022 > write: IOPS=16.8k, BW=2101MiB/s (2203MB/s)(123GiB/60001msec); 0 zone resets > slat (usec): min=5, max=125, avg=14.12, stdev=10.31 > clat (usec): min=306, max=65380, avg=460.78, stdev=506.24 > lat (usec): min=341, max=65389, avg=475.04, stdev=505.27 > > Which improves throughput and reduces lat. Just use blk_mq_complete_request_direct for callers that know they are not in interrupt context.
