The patch titled hrtimer: reduce calls to hrtimer_get_softirq_time() has been added to the -mm tree. Its filename is hrtimer-reduce-calls-to-hrtimer_get_softirq_time.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/SubmitChecklist when testing your code *** See http://www.zip.com.au/~akpm/linux/patches/stuff/added-to-mm.txt to find out what to do about this The current -mm tree may be found at http://userweb.kernel.org/~akpm/mmotm/ ------------------------------------------------------ Subject: hrtimer: reduce calls to hrtimer_get_softirq_time() From: Dimitri Sivanich <sivanich@xxxxxxx> It seems that hrtimer_run_queues() is calling hrtimer_get_softirq_time() more often than it needs to. This can cause frequent contention on systems with large numbers of processors/cores. With this patch, hrtimer_run_queues only calls hrtimer_get_softirq_time() if there is a pending timer in one of the hrtimer bases, and only once. This also combines hrtimer_run_queues() and the inline run_hrtimer_queue() into one function. Signed-off-by: Dimitri Sivanich <sivanich@xxxxxxx> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx> Cc: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx> Cc: Ingo Molnar <mingo@xxxxxxx> Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> --- kernel/hrtimer.c | 65 ++++++++++++++++++++++----------------------- 1 file changed, 33 insertions(+), 32 deletions(-) diff -puN kernel/hrtimer.c~hrtimer-reduce-calls-to-hrtimer_get_softirq_time kernel/hrtimer.c --- a/kernel/hrtimer.c~hrtimer-reduce-calls-to-hrtimer_get_softirq_time +++ a/kernel/hrtimer.c @@ -1238,51 +1238,52 @@ void hrtimer_run_pending(void) /* * Called from hardirq context every jiffy */ -static inline void run_hrtimer_queue(struct hrtimer_cpu_base *cpu_base, - int index) +void hrtimer_run_queues(void) { struct rb_node *node; - struct hrtimer_clock_base *base = &cpu_base->clock_base[index]; + struct hrtimer_cpu_base *cpu_base = &__get_cpu_var(hrtimer_bases); + struct hrtimer_clock_base *base; + int index; + int gst = 1; - if (!base->first) + if (hrtimer_hres_active()) return; - if (base->get_softirq_time) - base->softirq_time = base->get_softirq_time(); + for (index = 0; index < HRTIMER_MAX_CLOCK_BASES; index++) { + base = &cpu_base->clock_base[index]; - spin_lock(&cpu_base->lock); - - while ((node = base->first)) { - struct hrtimer *timer; - - timer = rb_entry(node, struct hrtimer, node); - if (base->softirq_time.tv64 <= timer->expires.tv64) - break; - - if (timer->cb_mode == HRTIMER_CB_SOFTIRQ) { - __remove_hrtimer(timer, base, HRTIMER_STATE_PENDING, 0); - list_add_tail(&timer->cb_entry, - &base->cpu_base->cb_pending); + if (!base->first) continue; + + if (gst) { + hrtimer_get_softirq_time(cpu_base); + gst = 0; } - __run_hrtimer(timer); - } - spin_unlock(&cpu_base->lock); -} + if (base->get_softirq_time) + base->softirq_time = base->get_softirq_time(); -void hrtimer_run_queues(void) -{ - struct hrtimer_cpu_base *cpu_base = &__get_cpu_var(hrtimer_bases); - int i; + spin_lock(&cpu_base->lock); - if (hrtimer_hres_active()) - return; + while ((node = base->first)) { + struct hrtimer *timer; - hrtimer_get_softirq_time(cpu_base); + timer = rb_entry(node, struct hrtimer, node); + if (base->softirq_time.tv64 <= timer->expires.tv64) + break; + + if (timer->cb_mode == HRTIMER_CB_SOFTIRQ) { + __remove_hrtimer(timer, base, + HRTIMER_STATE_PENDING, 0); + list_add_tail(&timer->cb_entry, + &base->cpu_base->cb_pending); + continue; + } - for (i = 0; i < HRTIMER_MAX_CLOCK_BASES; i++) - run_hrtimer_queue(cpu_base, i); + __run_hrtimer(timer); + } + spin_unlock(&cpu_base->lock); + } } /* _ Patches currently in -mm which might be from sivanich@xxxxxxx are hrtimer-reduce-calls-to-hrtimer_get_softirq_time.patch sgi-altix-mmtimer-allow-larger-number-of-timers-per-node.patch sgi-altix-mmtimer-allow-larger-number-of-timers-per-node-fix.patch sgi-altix-mmtimer-allow-larger-number-of-timers-per-node-fix-2.patch -- To unsubscribe from this list: send the line "unsubscribe mm-commits" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html