On Thu, Apr 27, 2017 at 1:43 AM, Lucas Stach <l.stach@xxxxxxxxxxxxxx> wrote:
> Am Mittwoch, den 26.04.2017, 14:11 -0700 schrieb Tim Harvey:
>> On Wed, Apr 26, 2017 at 11:15 AM, Lucas Stach <l.stach@xxxxxxxxxxxxxx> wrote:
>> > The DWC host does not support legacy PCI IRQs and MSIs at the same time,
>> > so we need to fall back to using only legacy IRQs if there is a chance
>> > that multiple devices with differing MSI capabilities are connected to
>> > the host. The only configuration where MSIs can be safely used is when
>> > the device below the host bridge is not a bridge, i.e. the only device
>> > connected to this host.
>> >
>> > By disallowing MSI allocation when multiple devices might be attached
>> > we get those configurations in a working state. The only configurations
>> > that depend on MSIs being available, that I am aware of, are some
>> > embedded devices with a PCIe attached FPGA, that is incapable of
>> > generating PCI legacy IRQs. Those are kept working by allowing MSIs
>> > when only a single device is attached to the host.
>> >
>> > Signed-off-by: Lucas Stach <l.stach@xxxxxxxxxxxxxx>
>> > ---
>> > drivers/pci/dwc/pcie-designware-host.c | 24 ++++++++++++++++++++++--
>> > drivers/pci/dwc/pcie-designware.h | 1 +
>> > 2 files changed, 23 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
>> > index 5ba334938b52..be2b2762c944 100644
>> > --- a/drivers/pci/dwc/pcie-designware-host.c
>> > +++ b/drivers/pci/dwc/pcie-designware-host.c
>> > @@ -92,6 +92,8 @@ void dw_pcie_msi_init(struct pcie_port *pp)
>> > (u32)(msi_target & 0xffffffff));
>> > dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_HI, 4,
>> > (u32)(msi_target >> 32 & 0xffffffff));
>> > +
>> > + pp->msi_disabled = false;
>> > }
>> >
>> > static void dw_pcie_msi_clear_irq(struct pcie_port *pp, int irq)
>> > @@ -136,9 +138,11 @@ static void dw_pcie_msi_set_irq(struct pcie_port *pp, int irq)
>> > static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
>> > {
>> > int irq, pos0, i;
>> > - struct pcie_port *pp;
>> > + struct pcie_port *pp = (struct pcie_port *)msi_desc_to_pci_sysdata(desc);
>> > +
>> > + if (pp->msi_disabled)
>> > + return -EINVAL;
>> >
>> > - pp = (struct pcie_port *)msi_desc_to_pci_sysdata(desc);
>> > pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
>> > order_base_2(no_irqs));
>> > if (pos0 < 0)
>> > @@ -410,6 +414,22 @@ int dw_pcie_host_init(struct pcie_port *pp)
>> > goto error;
>> > }
>> >
>> > + /*
>> > + * The DWC host does not support legacy PCI IRQs and MSIs at the same
>> > + * time, so we need to fall back to using only legacy IRQs if there is
>> > + * a chance that multiple devices with differing MSI capabilities are
>> > + * connected to the host. The only configuration where MSIs can be
>> > + * safely used is when the device below the host bridge is not a bridge,
>> > + * i.e. the only device connected to this host.
>> > + */
>> > + child = list_first_entry(&bus->children, struct pci_bus, node);
>> > + if (!list_empty(&child->devices)) {
>> > + struct pci_dev *dev = list_first_entry(&child->devices,
>> > + struct pci_dev, bus_list);
>> > + if (pci_is_bridge(dev))
>> > + pp->msi_disabled = true;
>> > + }
>> > +
>> > if (pp->ops->scan_bus)
>> > pp->ops->scan_bus(pp);
>> >
>> > diff --git a/drivers/pci/dwc/pcie-designware.h b/drivers/pci/dwc/pcie-designware.h
>> > index cd3b8713fe50..741bc31f8947 100644
>> > --- a/drivers/pci/dwc/pcie-designware.h
>> > +++ b/drivers/pci/dwc/pcie-designware.h
>> > @@ -140,6 +140,7 @@ struct pcie_port {
>> > struct irq_domain *irq_domain;
>> > unsigned long msi_data;
>> > DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
>> > + bool msi_disabled;
>> > };
>> >
>> > struct dw_pcie_ops {
>> > --
>> > 2.11.0
>> >
>>
>> Lucas,
>>
>> Legacy interrupts are still not firing with this.
>
> Can you provide me the output of "cat /proc/interrupts" and "lspci -t"
> on your system? I've tested this change on the GW54xx and legacy IRQs
> seemed to work there.
>
Lucas,
Right, I should have given you more detail. The legacy interrupts are
firing for endpoints through a bridge, but not for boards without a
bridge:
GW54xx + ath9k:
root@ventana:~# echo $(cat /proc/device-tree/model)
Gateworks Ventana i.MX6 Dual/Quad GW54XX
root@ventana:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 4343 1141 2358 1998 GIC-0 29 Edge twd
17: 91 79 76 65 GPC 55 Level
i.MX Timer Tick
18: 93375 0 0 0 GPC 13 Level mxs-dma
19: 21611 0 0 0 GPC 15 Level bch
20: 0 0 0 0 GPC 115 Level
120000.hdmi
21: 0 0 0 0 GPC 9 Level
130000.gpu
22: 0 0 0 0 GPC 10 Level
134000.gpu
24: 3 0 0 0 GPC 120 Level
mx6-pcie-msi, ath9k
^^^^ correct legacy irq and its firing (3 interrupts)
26: 0 0 0 0 GPC 32 Level
200c000.ecspi
27: 0 0 0 0 GPC 26 Level
2020000.serial
28: 0 0 0 0 GPC 46 Level
2028000.ssi
29: 0 0 0 0 GPC 47 Level
202c000.ssi
30: 0 0 0 0 GPC 50 Level
2034000.asrc
31: 0 0 0 0 GPC 12 Level
2040000.vpu
49: 0 0 0 0 GPC 81 Level
20c0000.wdog
50: 0 0 0 0 GPC 49 Level
imx_thermal
55: 0 0 0 0 GPC 19 Level
rtc alarm
61: 2 0 0 0 GPC 2 Level sdma
62: 0 0 0 0 GPC 43 Level
2184000.usb
63: 30 0 0 0 GPC 40 Level
2184200.usb
64: 1699 0 0 0 GIC-0 150 Level
2188000.ethernet
65: 0 0 0 0 GIC-0 151 Level
2188000.ethernet
66: 0 0 0 0 GPC 24 Level mmc0
67: 30 0 0 0 GPC 36 Level
21a0000.i2c
68: 92 0 0 0 GPC 37 Level
21a4000.i2c
69: 298 0 0 0 GPC 38 Level
21a8000.i2c
70: 0 0 0 0 GPC 18 Level vdoa
71: 500 0 0 0 GPC 27 Level
21e8000.serial
72: 0 0 0 0 GPC 30 Level
21f4000.serial
75: 0 0 0 0 GPC 39 Level
ahci-imx[2200000.sata]
76: 0 0 0 0 GPC 11 Level
2204000.gpu
271: 0 0 0 0 gpio-mxc 0 Edge
2198000.usdhc cd
335: 0 0 0 0 GPC 123 Level
PCIe PME, aerdrv
338: 0 0 0 0 IPU 457 Edge
2400000.ipu
339: 0 0 0 0 IPU 451 Edge
2400000.ipu
340: 0 0 0 0 IPU 457 Edge
2800000.ipu
341: 0 0 0 0 IPU 451 Edge
2800000.ipu
342: 1029 1 1 0 IPU 23 Edge imx_drm
343: 0 0 0 0 IPU 28 Edge imx_drm
344: 0 0 0 0 IPU 23 Edge imx_drm
345: 0 0 0 0 IPU 28 Edge imx_drm
346: 2 0 0 0 GPC 105 Level
2101000.jr0
347: 0 0 0 0 GPC 106 Level
2102000.jr1
IPI0: 0 0 0 0 CPU wakeup interrupts
IPI1: 25 36 15 22 Timer broadcast interrupts
IPI2: 862 8482 4490 4783 Rescheduling interrupts
IPI3: 36 29 41 44 Function call interrupts
IPI4: 0 0 0 0 CPU stop interrupts
IPI5: 229 174 430 250 IRQ work interrupts
IPI6: 0 0 0 0 completion interrupts
Err: 0
root@ventana:~# lspci -t
-[0000:00]---00.0-[01]--+-00.0-[02-09]--+-01.0-[03]--
| +-04.0-[04]--
| +-05.0-[05]--
| +-06.0-[06]--
| +-07.0-[07]----00.0
| +-08.0-[08]----00.0
| \-09.0-[09]--
\-00.1
GW51xx + ath9k (no bridge):
root@ventana:~# echo $(cat /proc/device-tree/model)
Gateworks Ventana i.MX6 Dual/Quad GW51XX
root@ventana:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 3818 991 1747 2665 GIC-0 29 Edge twd
17: 65 90 57 60 GPC 55 Level
i.MX Timer Tick
18: 39451 0 0 0 GPC 13 Level mxs-dma
19: 10397 0 0 0 GPC 15 Level bch
20: 0 0 0 0 GPC 115 Level
120000.hdmi
21: 0 0 0 0 GPC 9 Level
130000.gpu
22: 0 0 0 0 GPC 10 Level
134000.gpu
24: 0 0 0 0 GPC 120 Level
mx6-pcie-msi
26: 0 0 0 0 GPC 26 Level
2020000.serial
27: 0 0 0 0 GPC 50 Level
2034000.asrc
28: 0 0 0 0 GPC 12 Level
2040000.vpu
44: 0 0 0 0 GPC 49 Level
imx_thermal
49: 0 0 0 0 GPC 19 Level
rtc alarm
55: 0 0 0 0 GPC 2 Level sdma
56: 0 0 0 0 GPC 43 Level
2184000.usb
57: 0 0 0 0 GPC 40 Level
2184200.usb
58: 2186 0 0 0 GIC-0 150 Level
2188000.ethernet
59: 0 0 0 0 GIC-0 151 Level
2188000.ethernet
60: 30 0 0 0 GPC 36 Level
21a0000.i2c
61: 61 0 0 0 GPC 37 Level
21a4000.i2c
62: 0 0 0 0 GPC 38 Level
21a8000.i2c
63: 0 0 0 0 GPC 18 Level vdoa
64: 328 0 0 0 GPC 27 Level
21e8000.serial
65: 0 0 0 0 GPC 28 Level
21ec000.serial
66: 0 0 0 0 GPC 30 Level
21f4000.serial
69: 0 0 0 0 GPC 11 Level
2204000.gpu
80: 0 0 0 0 gpio-mxc 8 Level ltc3676
296: 0 0 0 0 PCI-MSI 0 Edge
PCIe PME, aerdrv
328: 0 0 0 0 GPC 123 Level ath9k
^^^^ correct legacy interrupt, but not firing (0 interrupts)
329: 0 0 0 0 IPU 457 Edge
2400000.ipu
330: 0 0 0 0 IPU 451 Edge
2400000.ipu
331: 0 0 0 0 IPU 457 Edge
2800000.ipu
332: 0 0 0 0 IPU 451 Edge
2800000.ipu
333: 0 0 0 0 IPU 23 Edge imx_drm
334: 0 0 0 0 IPU 28 Edge imx_drm
335: 0 0 0 0 IPU 23 Edge imx_drm
336: 0 0 0 0 IPU 28 Edge imx_drm
337: 2 0 0 0 GPC 105 Level
2101000.jr0
338: 0 0 0 0 GPC 106 Level
2102000.jr1
IPI0: 0 0 0 0 CPU wakeup interrupts
IPI1: 22 18 23 18 Timer broadcast interrupts
IPI2: 1016 5415 18358 6131 Rescheduling interrupts
IPI3: 40 44 34 50 Function call interrupts
IPI4: 0 0 0 0 CPU stop interrupts
IPI5: 455 152 326 609 IRQ work interrupts
IPI6: 0 0 0 0 completion interrupts
Err: 0
root@ventana:~# lspci -t
-[0000:00]---00.0-[01]----00.0
A GW51xx + ath10k (no bridge, device requesting MSI interrupts) looks
appropriate with the patch (uses MSI):
root@ventana:~# echo $(cat /proc/device-tree/model)
Gateworks Ventana i.MX6 Dual/Quad GW51XX
root@ventana:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 1759 1224 771 2368 GIC-0 29 Edge twd
17: 38 88 71 62 GPC 55 Level
i.MX Timer Tick
18: 39627 0 0 0 GPC 13 Level mxs-dma
19: 10451 0 0 0 GPC 15 Level bch
20: 0 0 0 0 GPC 115 Level
120000.hdmi
21: 0 0 0 0 GPC 9 Level
130000.gpu
22: 0 0 0 0 GPC 10 Level
134000.gpu
24: 78 0 0 0 GPC 120 Level
mx6-pcie-msi
26: 0 0 0 0 GPC 26 Level
2020000.serial
27: 0 0 0 0 GPC 50 Level
2034000.asrc
28: 0 0 0 0 GPC 12 Level
2040000.vpu
44: 0 0 0 0 GPC 49 Level
imx_thermal
49: 0 0 0 0 GPC 19 Level
rtc alarm
55: 0 0 0 0 GPC 2 Level sdma
56: 0 0 0 0 GPC 43 Level
2184000.usb
57: 0 0 0 0 GPC 40 Level
2184200.usb
58: 806 0 0 0 GIC-0 150 Level
2188000.ethernet
59: 0 0 0 0 GIC-0 151 Level
2188000.ethernet
60: 30 0 0 0 GPC 36 Level
21a0000.i2c
61: 61 0 0 0 GPC 37 Level
21a4000.i2c
62: 0 0 0 0 GPC 38 Level
21a8000.i2c
63: 0 0 0 0 GPC 18 Level vdoa
64: 201 0 0 0 GPC 27 Level
21e8000.serial
65: 0 0 0 0 GPC 28 Level
21ec000.serial
66: 0 0 0 0 GPC 30 Level
21f4000.serial
69: 0 0 0 0 GPC 11 Level
2204000.gpu
80: 0 0 0 0 gpio-mxc 8 Level ltc3676
296: 0 0 0 0 PCI-MSI 0 Edge
PCIe PME, aerdrv
297: 78 0 0 0 PCI-MSI 1 Edge
ath10k_pci
^^^^ MSI interrupt and its firing (78 interrupts)
329: 0 0 0 0 IPU 457 Edge
2400000.ipu
330: 0 0 0 0 IPU 451 Edge
2400000.ipu
331: 0 0 0 0 IPU 457 Edge
2800000.ipu
332: 0 0 0 0 IPU 451 Edge
2800000.ipu
333: 0 0 0 0 IPU 23 Edge imx_drm
334: 0 0 0 0 IPU 28 Edge imx_drm
335: 0 0 0 0 IPU 23 Edge imx_drm
336: 0 0 0 0 IPU 28 Edge imx_drm
337: 2 0 0 0 GPC 105 Level
2101000.jr0
338: 0 0 0 0 GPC 106 Level
2102000.jr1
IPI0: 0 0 0 0 CPU wakeup interrupts
IPI1: 12 24 13 20 Timer broadcast interrupts
IPI2: 905 7340 17490 6554 Rescheduling interrupts
IPI3: 50 49 34 48 Function call interrupts
IPI4: 0 0 0 0 CPU stop interrupts
IPI5: 192 160 60 29 IRQ work interrupts
IPI6: 0 0 0 0 completion interrupts
Err: 0
root@ventana:~# lspci -t
-[0000:00]---00.0-[01]----00.0
root@ventana:~# grep ath /proc/interrupts
297: 78 0 0 0 PCI-MSI 1 Edge
ath10k_pci
Regards,
Tim
