On Sat, 2012-10-06 at 14:34 +0200, Andrea Arcangeli wrote:
> Hi Mike,
Greetings,
> On Sat, Oct 06, 2012 at 04:39:54AM +0200, Mike Galbraith wrote:
> I think you just found a mistake.
>
> So disabling wake_affine if the wakeup CPU was on a remote NODE (only
> in that case it was turned off), meant sd_affine couldn't be turned on
> and for certain wakeups select_idle_sibling wouldn't run (rendering
> pointless some of my logic in select_idle_sibling).
Well, it still looks a bit bent to me no matter how I tilt my head.
/*
* If both cpu and prev_cpu are part of this domain,
* cpu is a valid SD_WAKE_AFFINE target.
*/
if (want_affine && (tmp->flags & SD_WAKE_AFFINE) &&
cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) {
affine_sd = tmp;
want_affine = 0;
}
Disabling when waker/wakee are cross node makes sense to me as a cycle
saver. If you have (SMT), MC and NODE domains, waker/wakee are cross
node, spans don't intersect, affine_sd remains NULL, the whole traverse
becomes a waste of cycles. If WAKE_BALANCE is enabled, we'll do that
instead (which pgbench and ilk should like methinks).
> > I measured the 1 in 1:N pgbench very much preferring mobility. The N,
> > dunno, but I don't imagine a large benefit for making them sticky
> > either. Hohum, numbers will tell the tale.
>
> Mobility on non-NUMA is an entirely different matter than mobility
> across NUMA nodes. Keep in mind there are tons of CPUs intra-node too
> so the mobility intra node may be enough. But I don't know exactly
> what the mobiltiy requirements of pgbench are so I can't tell for sure
> and I fully agree we should collect numbers.
Yeah, that 1:1 vs 1:N, load meets anti-load thing is kinda interesting.
Tune for one, you may well annihilate the other. Numbers required.
I think we need to detect and react accordingly. If that nasty little 1
bugger is doing a lot of work, it's very special, so I don't think you
making him sticky can help any more than me taking away wakeup options..
both remove latency reducing options from a latency dominated load.
But numbers talk, pondering (may = BS) walks, so I'm outta here :)
-Mike
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>