On Wed, Mar 20, 2024 at 10:34:46AM +0800, Ming Lei wrote: > Kernel parameter of `isolcpus=` or 'nohz_full=' are used to isolate CPUs > for specific task, and it isn't expected to let block IO disturb these CPUs. > blk-mq kworker shouldn't be scheduled on isolated CPUs. Also if isolated > CPUs is run for blk-mq kworker, long block IO latency can be caused. > > Kernel workqueue only respects CPU isolation for WQ_UNBOUND, for bound > WQ, the responsibility is on user because CPU is specified as WQ API > parameter, such as mod_delayed_work_on(cpu), queue_delayed_work_on(cpu) > and queue_work_on(cpu). > > So not run blk-mq kworker on isolated CPUs by removing isolated CPUs > from hctx->cpumask. Meantime use queue map to check if all CPUs in this > hw queue are offline instead of hctx->cpumask, this way can avoid any > cost in fast IO code path, and is safe since hctx->cpumask are only > used in the two cases. > > Cc: Tim Chen <tim.c.chen@xxxxxxxxxxxxxxx> > Cc: Juri Lelli <juri.lelli@xxxxxxxxxx> > Cc: Andrew Theurer <atheurer@xxxxxxxxxx> > Cc: Joe Mario <jmario@xxxxxxxxxx> > Cc: Sebastian Jug <sejug@xxxxxxxxxx> > Cc: Frederic Weisbecker <frederic@xxxxxxxxxx> > Cc: Bart Van Assche <bvanassche@xxxxxxx> > Cc: Tejun Heo <tj@xxxxxxxxxx> > Tested-by: Joe Mario <jmario@xxxxxxxxxx> > Signed-off-by: Ming Lei <ming.lei@xxxxxxxxxx> > --- > V4: > - improve comment & commit log as suggested by Tim Hello Jens, Tejun and Guys, This patch fixes one issue in OpenShift low latency environment, I appreciate you may take a look at the patch and merge it if you are fine. Thanks, Ming
