Hello Bjorn, Many thanks for your quick answer. Here are what you asked for: 1/ running 4.16.0-rc1 doesn't change anything. Issue still here... 2/ you'll find a "git diff" patch that contains the changes to switch the driver into full MSI mode attached to that e-mail. Here are also some other logs: $ uname -r 4.16.0-rc1 $ dmesg | grep peak_pciefd [ 4022.170354] peak_pciefd 0000:0b:00.0: 4x CAN-FD PCAN-PCIe FPGA v3.2.1: [ 4022.170456] peak_pciefd 0000:0b:00.0: MSI[1..4] enabling status: 4 [ 4022.170653] peak_pciefd 0000:0b:00.0: can0 at reg_base=0x00000000aab55a4d irq=70 [ 4022.170791] peak_pciefd 0000:0b:00.0: can1 at reg_base=0x0000000015bb0963 irq=71 [ 4022.170957] peak_pciefd 0000:0b:00.0: can2 at reg_base=0x000000004f07a1fc irq=72 [ 4022.171085] peak_pciefd 0000:0b:00.0: can3 at reg_base=0x00000000da023608 irq=73 When a canX interface is set to UP (see also $ ip link help can) then the IRQ is requested. It is freed when the interface goes back to DOWN. Setting UP both can0 and can1: $ sudo ip link set can0 up type can bitrate 500000 $ sudo ip link set can1 up type can bitrate 500000 $ cat /proc/interrupts | grep peak 70: 0 0 0 0 3141 0 0 0 0 1 0 0 IR-PCI-MSI 5767168-edge peak_pciefd 71: 1 0 0 0 0 0 0 0 0 2602 0 0 IR-PCI-MSI 5767169-edge peak_pciefd (You can notice that host is 12 cores CPU. CAN interrupts are each handled by different cores.) Setting DOWN can0 (=> issue): $ sudo ip link set can0 down $ cat /proc/interrupts | grep peak 71: 1 0 0 0 0 0 0 0 0 2602 0 0 IR-PCI-MSI 5767169-edge peak_pciefd Note that IR-PCI-MSI controller is being used, because of CONFIG_IRQ_REMAP. According to arch/x86/kernel/apic/msi.c, this controller belongs to a domain with MSI_FLAG_MULTI_PCI_MSI set. Regards, > -----Original Message----- > From: Bjorn Helgaas <helgaas@xxxxxxxxxx> > Sent: jeudi 22 février 2018 16:20 > To: Stéphane Grosjean <s.grosjean@xxxxxxxxxxxxxxx> > Cc: linux-pci@xxxxxxxxxxxxxxx; Marc Zyngier <marc.zyngier@xxxxxxx>; > Thomas Gleixner <tglx@xxxxxxxxxxxxx> > Subject: Re: free_irq(IRQ#0) on PCIe device with MSI enabled issue... > > [+cc Marc, Thomas] > > Hi Stéphane, > > Thanks for your email! > > On Thu, Feb 22, 2018 at 08:23:52AM +0000, Stéphane Grosjean wrote: > > I've been writing some drivers for Linux-can in the past years and > > want to add support of MSI in one of them (see > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drive > rs/net/can/peak_canfd). > > > > This device driver works with legacy shared INTx over a CAN PCIe > > device. I'd like to switch to a full MSI usage of this device, which > > proposes up to 4 vectors (this device is proposed with 1, 2 or 4 CAN > > FD ports). > > > > I'm running x86_64 Kernel 4.13.0-32 from Ubuntu 17.10. Everything > > looks ok as I successfully get the 4 consecutive IRQ levels. My > > problem is this one: as soon as the driver free_irq() the FISRT IRQ > > level, then it does not receive any interrupt from the device anymore, > > for all other 3 IRQs... On the contrary, when the driver > > free_irq() one of the 3 other IRQ (instead of the 1st one), then there > > is no problem and all other interrupts (including the first IRQ > > vector) are working as they should. > > Can you share the actual driver code you're running? The peak_canfd.c in > v4.16-rc1 doesn't have the MSI code in it, so I can't see exactly what you're > doing. > > The fact that it does work on ARM suggests there's some difference in the > arch code. I don't know enough about this area to figure it out from scratch, > but maybe working through the actual code would help. > > It would be ideal if you could reproduce the problem on v4.16-rc1, in case it's > something fixed since v4.13. > > > I contact you because this "issue" seems to be arch dependent: > > running the same driver and the same board on another arch (ARM), then > > everything works like a charm: the driver can free_irq() any IRQ in > > any order without disturbing the other ones. But running on x86, > > free_irq() on the FIRST MSI seems to (also) free all the other ones... > > > > Note that the driver set up MSI as described in the last version of > > Documentation/PCI/MSI-HOWTO.txt. -- PEAK-System Technik GmbH Sitz der Gesellschaft Darmstadt - HRB 9183 Geschaeftsfuehrung: Alexander Gach / Uwe Wilhelm --
Attachment:
0001-can-peak-canfd-add-MSI-support.patch
Description: 0001-can-peak-canfd-add-MSI-support.patch
