On Sun, Jul 10, 2022 at 01:38:01AM +0000, Joel Fernandes wrote: > On Fri, Jul 08, 2022 at 03:45:14PM -0700, Paul E. McKenney wrote: > > On Fri, Jul 08, 2022 at 04:17:30AM +0000, Joel Fernandes wrote: > > > On Sat, Jun 25, 2022 at 08:12:06PM -0700, Paul E. McKenney wrote: > > > > On Wed, Jun 22, 2022 at 10:50:53PM +0000, Joel Fernandes (Google) wrote: > > > > > > > > > > Hello! > > > > > Please find the next improved version of call_rcu_lazy() attached. The main > > > > > difference between the previous version is that it is now using bypass lists, > > > > > and thus handling rcu_barrier() and hotplug situations, with some small changes > > > > > to those parts. > > > > > > > > > > I also don't see the TREE07 RCU stall from v1 anymore. > > > > > > > > > > In the v1, we some numbers below (testing on v2 is in progress). Rushikesh, > > > > > feel free to pull these patches into your tree. Just to note, you will also > > > > > need to pull the call_rcu_lazy() user patches from v1. I have dropped in this > > > > > series, just to make the series focus on the feature code first. > > > > > > > > > > Following are power savings we see on top of RCU_NOCB_CPU on an Intel platform. > > > > > The observation is that due to a 'trickle down' effect of RCU callbacks, the > > > > > system is very lightly loaded but constantly running few RCU callbacks very > > > > > often. This confuses the power management hardware that the system is active, > > > > > when it is in fact idle. > > > > > > > > > > For example, when ChromeOS screen is off and user is not doing anything on the > > > > > system, we can see big power savings. > > > > > Before: > > > > > Pk%pc10 = 72.13 > > > > > PkgWatt = 0.58 > > > > > CorWatt = 0.04 > > > > > > > > > > After: > > > > > Pk%pc10 = 81.28 > > > > > PkgWatt = 0.41 > > > > > CorWatt = 0.03 > > > > > > > > So not quite 30% savings in power at the package level? Not bad at all! > > > > > > Yes this is the package residency amount, not the amount of power. This % is > > > not power. > > > > So what exactly is PkgWatt, then? If you can say. That is where I was > > getting the 30% from. > > Its the total package power (SoC power) - so like not just the CPU but also > the interconnect, other controllers and other blocks in there. > > This output is from the turbostat program and the number is mentioned in the > manpage: > "PkgWatt Watts consumed by the whole package." > https://manpages.debian.org/testing/linux-cpupower/turbostat.8.en.html Are we back to about a 30% savings in power at the package level? ;-) Either way, please quantify your "big power savings" by calculating and stating a percentage decrease. > > > > > Further, when ChromeOS screen is ON but system is idle or lightly loaded, we > > > > > can see that the display pipeline is constantly doing RCU callback queuing due > > > > > to open/close of file descriptors associated with graphics buffers. This is > > > > > attributed to the file_free_rcu() path which this patch series also touches. > > > > > > > > > > This patch series adds a simple but effective, and lockless implementation of > > > > > RCU callback batching. On memory pressure, timeout or queue growing too big, we > > > > > initiate a flush of one or more per-CPU lists. > > > > > > > > It is no longer lockless, correct? Or am I missing something subtle? > > > > > > > > Full disclosure: I don't see a whole lot of benefit to its being lockless. > > > > But truth in advertising! ;-) > > > > > > Yes, you are right. Maybe a better way I could put it is it is "lock > > > contention less" :D > > > > Yes, "reduced lock contention" would be a good phrase. As long as you > > carefully indicate exactly what scenario with greater lock contention > > you are comparing to. > > > > But aren't you acquiring the bypass lock at about the same rate as it > > would be aquired without laziness? What am I missing here? > > You are right, why not I just drop the locking phrases from the summary. > Anyway the main win from this work is not related to locking. Sounds good! Thanx, Paul > thanks, > > - Joel > > > > > Thanx, Paul > > > > > > > Similar results can be achieved by increasing jiffies_till_first_fqs, however > > > > > that also has the effect of slowing down RCU. Especially I saw huge slow down > > > > > of function graph tracer when increasing that. > > > > > > > > > > One drawback of this series is, if another frequent RCU callback creeps up in > > > > > the future, that's not lazy, then that will again hurt the power. However, I > > > > > believe identifying and fixing those is a more reasonable approach than slowing > > > > > RCU down for the whole system. > > > > > > > > Very good! I have you down as the official call_rcu_lazy() whack-a-mole > > > > developer. ;-) > > > > > > :-D > > > > > > thanks, > > > > > > - Joel > > >
