On 3/7/22 11:10 PM, Bart Van Assche wrote: > On 3/7/22 16:39, Mike Christie wrote: >> +static bool iscsi_xmit_from_qc; >> +module_param_named(xmit_from_queuecommand, iscsi_xmit_from_qc, bool, 0644); >> +MODULE_PARM_DESC(xmit_from_queuecommand, "Set to true to try to xmit the task from the queuecommand callout. The default is false wihch will xmit the task from the iscsi_q workqueue."); > > s/wihch/which/ ? > > It may be hard for users to get the value of this parameter right. Has it been considered to make the iSCSI initiator select the proper mode depending on the load? I think the DPDK and SPDK software supports this. This is supported via user-level multithreading and by scaling to more CPU cores if required to achieve full performance. The problem is guessing what the user is doing and wants. Users just do a wide range of things for whatever reasons: 1. No tuning at all. Let iscsi, network and app run wild on all CPUs. 2. Pin iscsi to specific CPUs. Let the app run wild. Or pin the networking and let iscsi and app run wild. 3. Pin iscsi and network to a specific CPU. Let app run wild. 4. Pin everything. They all have their benefits drawbacks like if you put the network and iscsi on the same CPU (or put them on different ones), and run the app on a different one you get the highest performance for a lot of workloads. Abusing the CPUs overcome the loss of caching. In this case, you might not want to use xmit_from_queuecommand because we could be transmitting from the CPU the app is on and it might have wanted those CPU cycle for it's own work. However, you might have to put everything on the same CPUs because they can't interfere with other workloads. In that case you take the perf hit. In this case xmit_from_queuecommand=true works really well for you. Note, if you start with xmit_from_queuecommand=true and if the user is doing a workload that doesn't allow us to transmit from the queucommand then we do drop back to the old style of doing things. We also do that for the case where SCSI WRITEs have to be broken up. So I didn't make it dynamic or change the default so I wouldn't mess up existing users.
