On 21.02.24 14:44, Mathias Nyman wrote:
> On 21.2.2024 1.43, Randy Dunlap wrote:
>> On 2/20/24 15:41, Randy Dunlap wrote:
>>> {+ tglx]
>>> On 2/20/24 15:19, Mikhail Gavrilov wrote:
>>>> On Mon, Feb 19, 2024 at 2:41 PM Mikhail Gavrilov
>>>> <mikhail.v.gavrilov@xxxxxxxxx> wrote:
>>>> I spotted network performance regression and it turned out, this was
>>>> due to the network card getting other interrupt. It is a side effect
>>>> of commit 57e153dfd0e7a080373fe5853c5609443d97fa5a.
>>> That's a merge commit (AFAIK, maybe not so much). The commit in
>>> mainline is:
>>>
>>> commit f977f4c9301c
>>> Author: Niklas Neronin <niklas.neronin@xxxxxxxxxxxxxxx>
>>> Date: Fri Dec 1 17:06:40 2023 +0200
>>>
>>> xhci: add handler for only one interrupt line
>>>
>>>> Installing irqbalance daemon did not help. Maybe someone experienced
>>>> such a problem?
>>>
>>> Thomas, would you look at this, please?
>>>
>>> A network device and xhci (USB) driver are now sharing interrupts.
>>> This causes a large performance decrease for the networking device.
>
> Short recap:
Thx for that. As the 6.8 release is merely two or three weeks away while
a fix is nowhere near in sight yet (afaics!) I start to wonder if we
should consider a revert here and try reapplying the culprit in a later
cycle when this problem is fixed.
Mathias, would that be an option? Or is there still hope that we see a
fix for this regression before the release of 6.8?
Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.
#regzbot poke
> xhci (USB) and network device didn't share interrupts, or even interrupt
> the
> same CPU in either good or bad case.
>
> A change in how many interrupts xhci driver requests changed which CPU
> the network device interrupts.
>
> In the bad case Mikhail Gavrilovs network device was interrupting CPU0
> together with:
> - IR-IO-APIC 2-edge timer
> - IR-PCI-MSIX-0000:07:00.0 1-edge nvme1q1
>
> In the good case network device was interrupting CPU27 together with:
> - IR-PCI-MSIX-0000:04:00.0 27-edge nvme0q27
> - IR-PCI-MSIX-0000:07:00.0 28-edge nvme1q28
>
> Manually moving network device irq 87 from CPU0 to CPU23 helped.
> (echo 800000 > /proc/irq/87/smp_affinity)
>
> Thanks
> -Mathias
>
