On Thu, 06 Jan 2022 16:20:47 +0000, Pali Rohár <pali@xxxxxxxxxx> wrote: > > On Thursday 06 January 2022 15:55:11 Marc Zyngier wrote: > > On Thu, 06 Jan 2022 15:44:47 +0000, > > Pali Rohár <pali@xxxxxxxxxx> wrote: > > > > > > On Thursday 06 January 2022 15:28:20 Marc Zyngier wrote: > > > > On Wed, 05 Jan 2022 15:02:38 +0000, > > > > Pali Rohár <pali@xxxxxxxxxx> wrote: > > > > > > > > > > This adds support for legacy INTx interrupts received from other PCIe > > > > > devices and which are reported by a new INTx irq chip. > > > > > > > > > > With this change, kernel can distinguish between INTA, INTB, INTC and INTD > > > > > interrupts. > > > > > > > > > > Note that for this support, device tree files has to be properly adjusted > > > > > to provide "interrupts" or "interrupts-extended" property with intx > > > > > interrupt source, "interrupt-names" property with "intx" string and also > > > > > 'interrupt-controller' subnode must be defined. > > > > > > > > > > If device tree files do not provide these nodes then driver would work as > > > > > before. > > > > > > > > > > Signed-off-by: Pali Rohár <pali@xxxxxxxxxx> > > > > > --- > > > > > drivers/pci/controller/pci-mvebu.c | 182 +++++++++++++++++++++++++++-- > > > > > 1 file changed, 174 insertions(+), 8 deletions(-) > > > > > > > > > > diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c > > > > > index 1e90ab888075..04bcdd7b7a6d 100644 > > > > > --- a/drivers/pci/controller/pci-mvebu.c > > > > > +++ b/drivers/pci/controller/pci-mvebu.c > > > > > @@ -54,9 +54,10 @@ > > > > > PCIE_CONF_ADDR_EN) > > > > > #define PCIE_CONF_DATA_OFF 0x18fc > > > > > #define PCIE_INT_CAUSE_OFF 0x1900 > > > > > +#define PCIE_INT_UNMASK_OFF 0x1910 > > > > > +#define PCIE_INT_INTX(i) BIT(24+i) > > > > > #define PCIE_INT_PM_PME BIT(28) > > > > > -#define PCIE_MASK_OFF 0x1910 > > > > > -#define PCIE_MASK_ENABLE_INTS 0x0f000000 > > > > > +#define PCIE_INT_ALL_MASK GENMASK(31, 0) > > > > > #define PCIE_CTRL_OFF 0x1a00 > > > > > #define PCIE_CTRL_X1_MODE 0x0001 > > > > > #define PCIE_CTRL_RC_MODE BIT(1) > > > > > @@ -110,6 +111,10 @@ struct mvebu_pcie_port { > > > > > struct mvebu_pcie_window iowin; > > > > > u32 saved_pcie_stat; > > > > > struct resource regs; > > > > > + struct irq_domain *intx_irq_domain; > > > > > + struct irq_chip intx_irq_chip; > > > > > > > > Why is this structure per port? It really should be global. Printing > > > > the port number in the name isn't enough of a reason. > > > > > > Because each port has its own independent set of INTA-INTD > > > interrupts. > > > > That doesn't warrant a copy of an irq_chip structure that contains the > > exact same callbacks, and only differs by *a string*. And the use of > > this string is only to end-up in /proc/interrupts, which is totally > > pointless. > > > > > > > > > > + raw_spinlock_t irq_lock; > > > > > + int intx_irq; > > > > > }; > > > > > > > > > > static inline void mvebu_writel(struct mvebu_pcie_port *port, u32 val, u32 reg) > > > > > @@ -235,7 +240,7 @@ static void mvebu_pcie_setup_wins(struct mvebu_pcie_port *port) > > > > > > > > > > static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > > > > > { > > > > > - u32 ctrl, lnkcap, cmd, dev_rev, mask; > > > > > + u32 ctrl, lnkcap, cmd, dev_rev, unmask; > > > > > > > > > > /* Setup PCIe controller to Root Complex mode. */ > > > > > ctrl = mvebu_readl(port, PCIE_CTRL_OFF); > > > > > @@ -288,10 +293,30 @@ static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > > > > > /* Point PCIe unit MBUS decode windows to DRAM space. */ > > > > > mvebu_pcie_setup_wins(port); > > > > > > > > > > - /* Enable interrupt lines A-D. */ > > > > > - mask = mvebu_readl(port, PCIE_MASK_OFF); > > > > > - mask |= PCIE_MASK_ENABLE_INTS; > > > > > - mvebu_writel(port, mask, PCIE_MASK_OFF); > > > > > + /* Mask all interrupt sources. */ > > > > > + mvebu_writel(port, ~PCIE_INT_ALL_MASK, PCIE_INT_UNMASK_OFF); > > > > > + > > > > > + /* Clear all interrupt causes. */ > > > > > + mvebu_writel(port, ~PCIE_INT_ALL_MASK, PCIE_INT_CAUSE_OFF); > > > > > + > > > > > + if (port->intx_irq <= 0) { > > > > > + /* > > > > > + * When neither "summary" interrupt, nor "intx" interrupt was > > > > > + * specified in DT then unmask all legacy INTx interrupts as in > > > > > + * this case driver does not provide a way for masking and > > > > > + * unmasking of individual legacy INTx interrupts. In this case > > > > > + * all interrupts, including legacy INTx are reported via one > > > > > + * shared GIC source and therefore kernel cannot distinguish > > > > > + * which individual legacy INTx was triggered. These interrupts > > > > > + * are shared, so it should not cause any issue. Just > > > > > + * performance penalty as every PCIe interrupt handler needs to > > > > > + * be called when some interrupt is triggered. > > > > > + */ > > > > > + unmask = mvebu_readl(port, PCIE_INT_UNMASK_OFF); > > > > > + unmask |= PCIE_INT_INTX(0) | PCIE_INT_INTX(1) | > > > > > + PCIE_INT_INTX(2) | PCIE_INT_INTX(3); > > > > > + mvebu_writel(port, unmask, PCIE_INT_UNMASK_OFF); > > > > > > > > Maybe worth printing a warning here, so that the user knows they are > > > > on thin ice. > > > > > > Ok. I can add it here. Anyway, this is default current state without > > > this patch. > > > > > > > > + } > > > > > } > > > > > > > > > > static struct mvebu_pcie_port *mvebu_pcie_find_port(struct mvebu_pcie *pcie, > > > > > @@ -924,6 +949,109 @@ static struct pci_ops mvebu_pcie_ops = { > > > > > .write = mvebu_pcie_wr_conf, > > > > > }; > > > > > > > > > > +static void mvebu_pcie_intx_irq_mask(struct irq_data *d) > > > > > +{ > > > > > + struct mvebu_pcie_port *port = d->domain->host_data; > > > > > + irq_hw_number_t hwirq = irqd_to_hwirq(d); > > > > > + unsigned long flags; > > > > > + u32 unmask; > > > > > + > > > > > + raw_spin_lock_irqsave(&port->irq_lock, flags); > > > > > + unmask = mvebu_readl(port, PCIE_INT_UNMASK_OFF); > > > > > + unmask &= ~PCIE_INT_INTX(hwirq); > > > > > + mvebu_writel(port, unmask, PCIE_INT_UNMASK_OFF); > > > > > + raw_spin_unlock_irqrestore(&port->irq_lock, flags); > > > > > +} > > > > > + > > > > > +static void mvebu_pcie_intx_irq_unmask(struct irq_data *d) > > > > > +{ > > > > > + struct mvebu_pcie_port *port = d->domain->host_data; > > > > > + irq_hw_number_t hwirq = irqd_to_hwirq(d); > > > > > + unsigned long flags; > > > > > + u32 unmask; > > > > > + > > > > > + raw_spin_lock_irqsave(&port->irq_lock, flags); > > > > > + unmask = mvebu_readl(port, PCIE_INT_UNMASK_OFF); > > > > > + unmask |= PCIE_INT_INTX(hwirq); > > > > > + mvebu_writel(port, unmask, PCIE_INT_UNMASK_OFF); > > > > > + raw_spin_unlock_irqrestore(&port->irq_lock, flags); > > > > > +} > > > > > + > > > > > +static int mvebu_pcie_intx_irq_map(struct irq_domain *h, > > > > > + unsigned int virq, irq_hw_number_t hwirq) > > > > > +{ > > > > > + struct mvebu_pcie_port *port = h->host_data; > > > > > + > > > > > + irq_set_status_flags(virq, IRQ_LEVEL); > > > > > + irq_set_chip_and_handler(virq, &port->intx_irq_chip, handle_level_irq); > > > > > + irq_set_chip_data(virq, port); > > > > > + > > > > > + return 0; > > > > > +} > > > > > + > > > > > +static const struct irq_domain_ops mvebu_pcie_intx_irq_domain_ops = { > > > > > + .map = mvebu_pcie_intx_irq_map, > > > > > + .xlate = irq_domain_xlate_onecell, > > > > > +}; > > > > > + > > > > > +static int mvebu_pcie_init_irq_domain(struct mvebu_pcie_port *port) > > > > > +{ > > > > > + struct device *dev = &port->pcie->pdev->dev; > > > > > + struct device_node *pcie_intc_node; > > > > > + > > > > > + raw_spin_lock_init(&port->irq_lock); > > > > > + > > > > > + port->intx_irq_chip.name = devm_kasprintf(dev, GFP_KERNEL, > > > > > + "mvebu-%s-INTx", > > > > > + port->name); > > > > > > > > That's exactly what I really don't want to see. It prevents sharing of > > > > the irq_chip structure, and gets in the way of making it const in the > > > > future. Yes, I know that some drivers do that. I can't fix those, > > > > because /proc/interrupts is ABI. But I really don't want to see more > > > > of these. > > > > > > Well, I do not understand why it should be shared and with who. HW has N > > > independent IRQ chips for legacy interrupts. And each one will be > > > specified in DT per HW layout / design. > > > > If you have multiple ports, all the ports can share the irq_chip > > structure. Actually scratch that. They *MUST* share the structure. The > > only reason you're not sharing it is to be able to print this useless > > string in /proc/interrupts. > > What is the point of sharing one irq chip if HW has N independent irq > chips (for legacy interrupts)? I do not catch it yet. And I do not care > here for /proc/interrupts, so also I have not caught what do you mean be > last sentence with "the only reason". > > And I still do not see how it could even work to have just one irq chip > and one irq domain as each irq domain needs to know to which port it > belongs, so it can mask/unmask interrupts from correct port. Also > initialization of domain is taking DT node and for each port it is > different. > > So I'm somehow confused here... > > The improvement in this patch is to be able to mask INTA interrupts on > port 1 and let INTA interrupts unmasked on port 2 if there drivers are > interested only for interrupts from device connected to port 2. > > And if all interrupts are going to be shared (again) then it does not > solve any problem. You are completely missing my point. I'm talking about data structures, you're talking about interrupts. You have this: struct mvebu_pcie_port { // Tons of stuff struct irq_chip intx_chip; }; What I want you to do is: struct mvebu_pcie_port { // Tons of stuff }; static struct irq_chip intx_chip = { .name = "INTx", .irq_mask = mvebu_pcie_intx_irq_mask, .irq_unmask = mvebu_pcie_intx_irq_unmask; }; That's it. No more, no less. M. -- Without deviation from the norm, progress is not possible.