Hi! I am measuring performance of the RCU consolidated vs RCU before the consolidation of flavors happened (just for fun and may be to talk about in a presentation). What I did is I limited the readers/writers in rcuperf to run on all but one CPU. And then on that one CPU, I had a thread doing a preempt-disable + busy-wait + preempt_enable in a loop. I was hoping the preempt disable busy-wait thread would stall the regular readers, and it did. But what I noticed is that grace periods take 100-200 milliseconds to finish instead of the busy-wait time of 5-10 ms that I set. On closer examination, it looks like even though the preempt_enable happens in my loop, the need-resched flag is not set even though the grace period is long over due. So the thread does not reschedule. For now, in my test I am just setting the need-resched flag manual after a busy wait. But I was thinking, can this really happen in real life? So, say a CPU is doing a lot of work in preempt_disable but is diligent enough to check need-resched flag periodically. I believe some spin-on-owner type locking primitives do this. Even though the thread is stalling the grace period, it has no clue because no one told it that a GP is in progress that is being held up. The tick interrupt for that thread returns rcu_need_deferred_qs() returns false during the preempt disable section. Can we do better for such usecases, such as even sending an IPI to the CPUs holding the Grace period? Or even upgrading the grace period to an expedited one if need be? Expedited grace periods did not have such issues. However I did notice that sometimes the Grace period would end not within 1 busy-wait duration but within 2. The distribution was strongly bi-modal to 1*busy-wait and 2*busy-wait durations for expedited tests. (This expedited test actually happened by accident, because the preempt-disable in my loop was delaying init enough that the whole test was running during init during which synchronize_rcu is upgraded to expedited). I am sorry if this is not a realistic real-life problem, but more a "doctor it hurts if I do this" problem as Steven once said ;-) I'll keep poking ;-) J.
