On 2/6/21 7:43 PM, Nitesh Narayan Lal wrote: > On 2/5/21 5:23 PM, Thomas Gleixner wrote: >> On Thu, Feb 04 2021 at 14:17, Nitesh Narayan Lal wrote: >>> On 2/4/21 2:06 PM, Marcelo Tosatti wrote: >>>>>> How about adding a new flag for isolcpus instead? >>>>>> >>>>> Do you mean a flag based on which we can switch the affinity mask to >>>>> housekeeping for all the devices at the time of IRQ distribution? >>>> Yes a new flag for isolcpus. HK_FLAG_IRQ_SPREAD or some better name. >>> Does sounds like a nice idea to explore, lets see what Thomas thinks about it. <snip> >>> When the affinity mask of the interrupt at the time when it is actually >>> requested contains an isolated CPU then nothing prevents the kernel from >>> steering it at an isolated CPU. But that has absolutely nothing to do >>> with that spreading thingy. >>> >>> The only difference which this change makes is the fact that the >>> affinity hint changes. Nothing else. >>> > Thanks for the detailed explanation. > > Before I posted this patch, I was doing some debugging on a setup where I > was observing some latency issues due to the iavf IRQs that were pinned on > the isolated CPUs. > > Based on some initial traces I had this impression that the affinity hint > or cpumask_local_spread was somehow playing a role in deciding the affinity > mask of these IRQs. Although, that does look incorrect after going through > your explanation. > For some reason, with a kernel that had this patch when I tried creating > VFs iavf IRQs always ended up on the HK CPUs. > > The reasoning for the above is still not very clear to me. I will investigate > this further to properly understand this behavior. > > After a little more digging, I found out why cpumask_local_spread change affects the general/initial smp_affinity for certain device IRQs. After the introduction of the commit: e2e64a932 genirq: Set initial affinity in irq_set_affinity_hint() For all the drivers that set hint, initial affinity is set based on the CPU retrieved from cpumask_local_spread. So in an environment where irqbalance is disabled, these device IRQs remain on the CPUs that are picked from cpumask_local_spread even though they are isolated. I think the commit message of the reverted patch should have covered this as well. -- Thanks Nitesh
