On 2022-09-21 13:54:43 [-1000], Tejun Heo wrote: > Hello, Hi, > On Thu, Sep 22, 2022 at 12:32:49AM +0200, Jason A. Donenfeld wrote: > > What are our options? Investigate queue_work_on() bottlenecks? Move back > > to the original pattern, but use raw spinlocks? Some thing else? > > I doubt it's queue_work_on() itself if it's called at very high frequency as > the duplicate calls would just fail to claim the PENDING bit and return but > if it's being called at a high frequency, it'd be waking up a kthread over > and over again, which can get pretty expensive. Maybe that ends competing > with softirqd which is handling net rx or sth? There is this (simplified): | if (new_count & MIX_INFLIGHT) | return; | | if (new_count < 1024 && !time_is_before_jiffies(fast_pool->last + HZ)) | return; | | fast_pool->count |= MIX_INFLIGHT; | queue_work_on(raw_smp_processor_id(), system_highpri_wq, &fast_pool->mix); at least 1k interrupts are needed and a second must pass before a worker will be scheduled. Oh wait. We need only one of both. So how many interrupts do we get per second? Is the regression coming from more than 1k interrupts in less then a second or a context switch each second? Because if it is a context switch every second then I am surprised to see a 10% performance drop in this case since should happen for other reasons, too unless the CPU is isolated. [ There isn't a massive claims of the PENDING bit or wakeups because fast_pool is per-CPU and due to the MIX_INFLIGHT bit. ] > So, yeah, I'd try something which doesn't always involve scheduling and a > context switch whether that's softirq, tasklet, or irq work. I probably am > mistaken but I thought RT kernel pushes irq handling to threads so that > these things can be handled sanely. Is this some special case? As Jason explained this part is invoked in the non-threaded part. > Thanks. Sebastian
