On Mon, 26 Jan 2009 21:20:22 +0100
Ingo Molnar <mingo@xxxxxxx> wrote:
>
> * Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> wrote:
>
> > On Mon, 26 Jan 2009 18:16:18 +0100
> > Ingo Molnar <mingo@xxxxxxx> wrote:
> >
> > >
> > > * Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> wrote:
> > >
> > > > > > Yet another kernel thread for each CPU. All because of some dung
> > > > > > way down in arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.c.
> > > > > >
> > > > > > Is there no other way?
> > > > >
> > > > > Perhaps, but this works. Trying to be clever got me into this mess in
> > > > > the first place.
> > > > >
> > > > > We could stop using workqueues and change work_on_cpu to create a
> > > > > thread every time, which would give it a new failure mode so I don't
> > > > > know that everyone could use it any more. Or we could keep a single
> > > > > thread around to do all the cpus, and duplicate much of the workqueue
> > > > > code.
> > > > >
> > > > > None of these options are appealing...
> > > >
> > > > Can we try harder please? 10 screenfuls of kernel threads in the ps
> > > > output is just irritating.
> > > >
> > > > How about banning the use of work_on_cpu() from schedule_work() handlers
> > > > and then fixing that driver somehow?
> > >
> > > Yes, but that's fundamentally fragile: anyone who happens to stick the
> > > wrong thing into keventd (and it's dead easy because schedule_work() is
> > > easy to use) will lock up work_on_cpu() users.
> > >
> >
> > --- a/kernel/workqueue.c~a
> > +++ a/kernel/workqueue.c
> > @@ -998,6 +998,8 @@ long work_on_cpu(unsigned int cpu, long
> > {
> > struct work_for_cpu wfc;
> >
> > + BUG_ON(current_is_keventd());
> > +
> > INIT_WORK(&wfc.work, do_work_for_cpu);
> > wfc.fn = fn;
> > wfc.arg = arg;
> > _
> >
> >
> > That wasn't so hard.
>
> What is the purpose of your change? I'm not sure you understood the
> problem.
Well. That's because I was forced to resort to guesswork.
> The problem is not with work_on_cpu() usage. The problem is:
>
> 1) holding locks while calling work_on_cpu()
>
> 2) same locks being taken by a worklet used by some other code
>
> work_on_cpu() really wants to serialize on its own workload only, not on
> the other stuff that might be sometimes be queued up in the keventd
> workqueue.
but but but, we fixed that ages ago, I think. But I don't see the code
there.
If we want to wait on a *particular* keventd work item then we
shouldn't wait on all the other queued ones.
- If it's currently running, wait on it
- If it isn't yet running, detach it from the queue and run it directly.
Maybe I'm thinking of a different subsystem, but I don't think so.
Maybe Oleg recalls what happened to that?
> > > work_on_cpu() is an important (and lowlevel enough) facility to be
> > > isolated from casual interaction like that.
> >
> > We have one single (known) caller in the whole kernel. This is not
> > worth adding another great pile of kernel threads for!
>
> i'd expect there to be more as part of the cpumask stack reduction
> patches that Rusty and Mike are working on.
>
> in any case it's a correctness issue: work_on_cpu() is a just as generic
> facility as on_each_cpu() - with the difference that it can handle
> blocking contexts too.
Well on_each_cpu() has restrictions. Can't all it with local
interrupts disabled. Can't call it (synchronously) while holding locks
which the callback takes.
> So if it's generic it ought to be implemented in a generic way - not a
> "dont use from any codepath that has a lock held that might occasionally
> also be held in a keventd worklet". (which is a totally unmaintainable
> proposition and which would just cause repeat bugs again and again.)
That's different. The core fault here lies in the keventd workqueue
handling code. If we're flushing work A then we shouldn't go and block
behind unrelated work B.
--
To unsubscribe from this list: send the line "unsubscribe cpufreq" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
