On Sun, May 28, 2006 at 12:20:30AM -0400, Kumba wrote:
> >If it's really, really not deadlocky to call smp_call_function() with
> >interrupts disabled at that time in the MIPS kernel bringup then I'd
> >suggest you should open-code an smp_call_function() and put a big comment
> >over it explaining why it's done this way, and why it isn't deadlocky.
MIPS does on_each_cpu() for cache maintenance. In the early stages before
interrupts are enabled there is only one CPU active anyway, so it's
perfectly safe - and obvious to the reader I hope - to use the local_*
variant of the cacheflushes. smp_call_function will yell loudly anyway
if called with interrupts disabled.
> ><tries to remember what the deadlock is>
> >
> >If CPU A is running smp_call_function() it's waiting for CPU B to run the
> >handler.
> >
> >But if CPU B is presently _also_ running smp_call_function(), it's waiting
> >for CPU A to run the handler.
> >
> >If either of those CPUs is waiting for the other with local interrupts
> >disabled, that CPU will never respond to the other CPU's IPI and they'll
> >deadlock.
Circumstances which don't apply during the early startup phase but
since the MIPS smp_call_function() does a WARN_ON(irqs_disabled()) I had
to clean that.
> The catch is, the system being affected here is strictly a UP machine.
> It's impossible to make an O2 go SMP. It seems that the disable call in
> the UP version of on_each_cpu() (which I assume is the #define macro) is
> what's causing this issue, since the machine comes to a halt in the dark
> void between function calls (i.e., that memset() I alluded to earlier)
>
> So I'm wondering, is there a way to see if the IRQ handlers have been
> installed already prior to disabling them, or is this more of a
> machine-specific oddity wherein the IRQ handlers need to be setup earlier
> (I don't even know if this is even possible/relevant to O2 systems)?
>
> It also seems this was affecting AMD Alchemy-based systems too. Other SGI
> machines are known to work fine, except Indy and Indigo2, as I haven't
> tested those yet.
IP27 is fine but it's SMP but I've already cleaned out all the early
calls to smp_call_function there were shown by the WARN() ages ago.
You can do it the same way, use this debugging version of on_each_cpu:
#define on_each_cpu(func,info,retry,wait) \
({ \
WARN_ON(irqs_disabled()); \
func(info); \
0; \
})
Ralf
