On 03/07/18 10:58, Marc Zyngier wrote: > On 03/07/18 08:01, takahiro.akashi@xxxxxxxxxx wrote: >> Marc, James, >> >> I'd like to re-ignite the discussion. >> >> On Sun, Jun 10, 2018 at 01:24:17PM +0100, Marc Zyngier wrote: >>> On Wed, 06 Jun 2018 12:37:02 +0100, >>> James Morse wrote: >>>> >>>> Hi Stefan, >>>> >>>> On 06/06/18 08:02, Stefan Wahren wrote: >>>>> Am 05.06.2018 um 19:46 schrieb James Morse: >>>>>> On 05/06/18 09:01, Petr Tesarik wrote: >>>>>>> I attached a hardware debugger and found >>>>>>> out that all CPU cores were stopped except one which was stuck in the >>>>>>> idle thread. It seems that irq_set_irqchip_state() may sleep, which is >>>>>>> definitely not safe after a kernel panic. >>>> >>>>>> I don't know much about irqchip stuff, but __irq_get_desc_lock() takes a >>>>>> raw_spin_lock(), and calls gic_irq_get_irqchip_state() which is just poking >>>>>> around in mmio registers, this should all be safe unless you re-entered the same >>>>>> code. >>>> >>>>>>> If I'm right, then this is broken in general, but I have only ever seen >>>>>>> it on RPi 3 Model B+ (even RPi3 Model B works fine), so the issue may >>>>>>> be more subtle. >>>> >>>>>> Is there a hardware difference around the interrupt controller on these? >>>> >>>>> No, but the RPi 3 B has a different USB network chip on board (smsc95xx, Fast >>>>> ethernet) instead of lan78xx (Gigabit ethernet). >>>> >>>> Bingo: its the lan78xx driver that is sleeping from the irqchip >>>> callbacks; The smsc95xx driver doesn't have a struct irq_chip, which >>>> is why the RPi-3-B doesn't do this. >>>> >>>> It may be valid for kdump to only teardown the 'root irqdomain' (if >>>> that even means anything). I assume these secondary irqchip's would >>>> have a summary-interrupt that goes to another irqchip. But I can't >>>> see a way to tell them apart.., >>> >>> There is none. A cascaded irqchip is just like a root irqchip, just >>> that its output line is connected to another irqchip. But we have no >>> easy way to identify the parent. Also, this particular driver looks >>> quite creative (it reinvents the wheel for chained interrupts -- see >>> intr_complete and lan78xx_status), meaning that even if we could have >>> a magic way of identify a chained irqchip, we'd miss that one. Broken. >>> >>>> I think we need to wait until after the merge window for Marc's >>>> wisdom on this! >>> >>> Overall, I can't think of an easy fix. We have a few options, but none >>> of them involve a centralised change: >>> >>> 1) We provide a reset infrastructure for irqchips, with an opt-in >>> mechanism. This involves changing the way we teardown irqs at >>> crash-time, and we'd then need some notion of reset ordering (think >>> of the layered ITS and GICv3, for example). >> >> Does this mean that all the irqchips have to be implemented with reset? > > No. Only those that want to be reset at kexec time. > >>> >>> 2) We provide a way to identify interrupts that are ultimately backed >>> by a root controller, which implies walking down the hierarchy for >> >> To be clear, from bottom to top (or root), right? > > I'm not sure I understand your question. The idea is to walk the > irq_data chain, until we hit a root irqchip. If we do hit one, we > deactivate/eoi/disable this interrupt. If we don't, we do nothing. > > This would avoid the above brokenness, and still ensures that no > interrupt reaches the CPU. > >> >>> each one of them. Fairly expensive, but minimal in way of changes >>> in the crash code. Requires a per-irqchip flag, but ordering comes >>> in for free. >>> >>> 3) We do the same as (2), but at the irqdomain level. Not sure that's >>> any better, and it may be even more complicated and bring back some >>> ordering issues. >> >> Do you think that the same thing may happen in case of pci/msi? >> I have no confidence but MSI has some kind of irq domain hierarchy. > > Anything can happen, as people implement their interrupt infrastructure > in weird and wonderful ways. So we need to be prepared for the worse. > > I've pushed 3 patches on a branch[1]. It is mostly untested, but it > should allow the above RPi3 disaster to cope with kexec. > > M. > > [1]: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=irq/root-irqchip > I threw the kernel on my RPi3+ model but I wasn't able to start the crash kernel. Unfortunately I don't have a JTAG adapter to check if it hangs for the same reason. Regards, Matthias _______________________________________________ kexec mailing list kexec@xxxxxxxxxxxxxxxxxxx http://lists.infradead.org/mailman/listinfo/kexec
