RE: [PATCH net-next v8 6/9] net: txgbe: Support GPIO to SFP socket

[Jiawen Wu <jiawenwu@xxxxxxxxxxxxxx>]

On Thursday, May 18, 2023 8:49 PM, Andrew Lunn wrote:
> > > I _think_ you are mixing upstream IRQs and downstream IRQs.
> > >
> > > Interrupts are arranged in trees. The CPU itself only has one or two
> > > interrupts. e.g. for ARM you have FIQ and IRQ. When the CPU gets an
> > > interrupt, you look in the interrupt controller to see what external
> > > or internal interrupt triggered the CPU interrupt. And that interrupt
> > > controller might indicate the interrupt came from another interrupt
> > > controller. Hence the tree structure. And each node in the tree is
> > > considered an interrupt domain.
> > >
> > > A GPIO controller can also be an interrupt controller. It has an
> > > upstream interrupt, going to the controller above it. And it has
> > > downstream interrupts, the GPIO lines coming into it which can cause
> > > an interrupt. And the GPIO interrupt controller is a domain.
> > >
> > > So what exactly does gpio_regmap_config.irq_domain mean? Is it the
> > > domain of the upstream interrupt controller? Is it an empty domain
> > > structure to be used by the GPIO interrupt controller? It is very
> > > unlikely to have anything to do with the SFP devices below it.
> >
> > Sorry, since I don't know much about interrupt,  it is difficult to understand
> > regmap-irq in a short time. There are many questions about regmap-irq.
> >
> > When I want to add an IRQ chip for regmap, for the further irq_domain,
> > I need to pass a parameter of IRQ, and this IRQ will be requested with handler:
> > regmap_irq_thread(). Which IRQ does it mean?
> 
> That is your upstream IRQ, the interrupt indicating one of your GPIO
> lines has changed state.
> 
> > In the previous code of using
> > devm_gpiochip_add_data(), I set the MSI-X interrupt as gpio-irq's parent, but
> > it was used to set chained handler only. Should the parent be this IRQ? I found
> > the error with irq_free_descs and irq_domain_remove when I remove txgbe.ko.
> 
> Do you have one MSI-X dedicated for GPIOs. Or is it your general MAC
> interrupt, and you need to read an interrupt controller register to
> determine it was GPIOs which triggered the interrupt?
> 
> If you are getting errors when removing the driver it means you are
> missing some level of undoing what us done in probe. Are you sure
> regmap_del_irq_chip() is being called on unload?
> 
> > As you said, the interrupt of each tree node has its domain. Can I understand
> > that there are two layer in the interrupt tree for MSI-X and GPIOs, and requesting
> > them separately is not conflicting? Although I thought so, but after I implement
> > gpio-regmap, SFP driver even could not find gpio_desc. Maybe I missed something
> > on registering gpio-regmap...
> 
> That is probably some sort of naming issue. You might want to add some
> prints in swnode_find_gpio() and gpiochip_find() to see what it is
> looking for vs what the name actually is.

It's true for the problem of name, but there is another problem. SFP driver has
successfully got gpio_desc, then it failed to get gpio_irq from gpio_desc (with error
return -517). I traced the function gpiod_to_irq():

	gc = desc->gdev->chip;
	offset = gpio_chip_hwgpio(desc);
	if (gc->to_irq) {
		int retirq = gc->to_irq(gc, offset);

		/* Zero means NO_IRQ */
		if (!retirq)
			return -ENXIO;

		return retirq;
	}

'gc->to_irq = gpiochip_to_irq' was set in [4]gpiochip_irqchip_add_domain().
So:

	static int gpiochip_to_irq(struct gpio_chip *gc, unsigned int offset)
	{
		struct irq_domain *domain = gc->irq.domain;

	#ifdef CONFIG_GPIOLIB_IRQCHIP
		/*
		 * Avoid race condition with other code, which tries to lookup
		 * an IRQ before the irqchip has been properly registered,
		 * i.e. while gpiochip is still being brought up.
		 */
		if (!gc->irq.initialized)
			return -EPROBE_DEFER;
	#endif

gc->irq.initialized is set to true at the end of [3]gpiochip_add_irqchip() only.
Firstly, it checks if irqchip is NULL:

	static int gpiochip_add_irqchip(struct gpio_chip *gc,
					struct lock_class_key *lock_key,
					struct lock_class_key *request_key)
	{
		struct fwnode_handle *fwnode = dev_fwnode(&gc->gpiodev->dev);
		struct irq_chip *irqchip = gc->irq.chip;
		unsigned int type;
		unsigned int i;

		if (!irqchip)
			return 0;

The result shows that it was NULL, so gc->irq.initialized = false.
Above all, return irq = -EPROBE_DEFER.

So let's sort the function calls. In chronological order, [1] calls [2], [2] calls
[3], then [1] calls [4]. The irq_chip was added to irq_domain->host_data->irq_chip
before [1]. But I don't find where to convert gpio_chip->irq.domain->host_data->irq_chip
to gpio_chip->irq.chip, it seems like it should happen after [4] ? But if it wants to use
'gc->to_irq' successfully, it should happen before [3]?

[1] gpio_regmap_register()
[2] gpiochip_add_data()
[3] gpiochip_add_irqchip()
[4] gpiochip_irqchip_add_domain()

I'm sorry that I described the problem in a confusing way, apologize if I missed
some code that caused this confusion.