On 3/8/22 6:53 PM, Ming Lei wrote: > On Mon, Mar 07, 2022 at 06:39:54PM -0600, Mike Christie wrote: >> The software iscsi driver's queuecommand can block and taking the extra >> hop from kblockd to its workqueue results in a performance hit. Allowing >> it to set BLK_MQ_F_BLOCKING and transmit from that context directly >> results in a 20-30% improvement in IOPs for workloads like: >> >> fio --filename=/dev/sdb --direct=1 --rw=randrw --bs=4k --ioengine=libaio >> --iodepth=128 --numjobs=1 >> >> and for all write workloads. > > This single patch shouldn't make any difference for iscsi, so please > make it as last one if performance improvement data is provided > in commit log. Ok. > > Also is there performance effect for other worloads? such as multiple > jobs? iscsi is SQ hardware, so if driver is blocked in ->queuecommand() > via BLK_MQ_F_BLOCKING, other contexts can't submit IO to scsi ML any more. If you mean multiple jobs running on the same connection/session then they are all serialized now. A connection can only do 1 cmd at a time. There's a big mutex around it in the network layer, so multiple jobs just suck no matter what. If you mean multiple jobs from different connection/sessions, then the iscsi code with this patchset blocks only because the network layer takes a mutex for a short time. We configure it to not block for things like socket space, memory allocations, we do zero copy IO normally, etc so it's quick. We also can do up to workqueues max_active limit worth of calls so other things can normally send IO. We haven't found a need to increase it yet.
