On 7/14/2022 4:51 PM, Paul E. McKenney wrote: > On Wed, Jul 13, 2022 at 09:32:32PM +0000, Joel Fernandes (Google) wrote: >> Hello! >> >> Please find the next improved version of call_rcu_lazy() attached. The main >> difference between the previous versions is that: >> - In v2 rcu_barrier is fixed to not hang (I found this to be due to a missing >> GP thread wakeup), now I am limiting this wake up only to rcu_barrier() as >> requested by Paul. >> - Fixed checkpatch and build robot issues. >> - Some more changes to 'lazy' parameter passing and consolidation of segcblist >> functions. >> - more testing via rcutorture and rcuscale. > > Thank you! What I am going to do is to pull these into an experimental > not-for-mainline branch and run the usual set of rcutorture tests. > I will then take a look at the patches. > >> Note that these tests were run on v2 patches, I am expecting similar power >> improvements however I've not yet tested power. >> >> Following are power savings we saw on top of RCU_NOCB_CPU on an Intel platform >> in v2. The observation is that due to a 'trickle down' effect of RCU >> callbacks, the system is very lightly loaded but constantly running few RCU >> callbacks very often. This confuses the power management hardware that the >> system is active, when it is in fact idle. >> >> For example, when ChromeOS screen is off and user is not doing anything on the >> system, we can see big power savings. >> Before: >> Pk%pc10 = 72.13 >> PkgWatt = 0.58 >> CorWatt = 0.04 >> >> After: >> Pk%pc10 = 81.28 >> PkgWatt = 0.41 >> CorWatt = 0.03 > > When you update these numbers, please explain what they all are and > evaluate them in the cover letter (or in the relevant patch's commit log). > For final submission, please also include some estimate of the variance. > For example, CorWatt might be essentially the same both before and after, > as in 0.035 and 0.034, or there might be a large difference, as in 0.044 > and 0.025. The 81.28 might be constant in all four digits (ha!), or it > might vary between (say) 80 and 83. And so on. > > Based on our earlier emails, my guess is that Pk%pc10 is the percent of > time that the system is in a low-power state (bigger is better), PkgWatt > is power consumed by the CPU chip (smaller is better), and CorWatt is > power consumed by the CPU core (again, smaller is better). > > It all might sound painfully obvious to you right now, but please have > pity on future readers of this series. Following is some power data I collected with Turbostat on an x86 ChromeOS ADL machine. The power seems better on the upstream kernel already versus my earlier tests, however I still see 5-8% power improvement with the patches. Rushikesh, Sitanshu, Neeraj, Vlad - could you test power with these as well? The following branch boots on ChromeOS ADL: https://github.com/joelagnel/linux-kernel/tree/chromeos-kernelupstream-5.19-rc4.aug22 These are output from Turbostat running: turbostat -S -s PkgWatt,CorWatt --interval 5 PkgWatt - summary of package power in Watts 5 second interval. CoreWatt - summary of core power in Watts 5 second interval. I could not get PC10% on the upstream kernel, it always shows 0. Maybe that is a turbostat version issue or something. +───────────+─────────────────────────+──────────+───+──────────────────────────+──────────+ | | Screen on idle (After) | | | Screen on idle (Before) | | +───────────+─────────────────────────+──────────+───+──────────────────────────+──────────+ | | PkgWatt | CorWatt | | PkgWatt | CorWatt | | | | | | | | | | 0.6100 | 0.0500 | | 0.6700 | 0.0500 | | | 0.5800 | 0.0400 | | 0.6200 | 0.0500 | | | 0.5800 | 0.0400 | | 0.6500 | 0.0500 | | | 0.6600 | 0.0800 | | 0.6200 | 0.0500 | | | 0.5900 | 0.0400 | | 0.6200 | 0.0500 | | | 0.5900 | 0.0400 | | 0.6400 | 0.0600 | | | 0.6000 | 0.0500 | | 0.6700 | 0.0600 | | | 0.6400 | 0.0600 | | 0.6200 | 0.0500 | | | 0.6000 | 0.0500 | | 0.6200 | 0.0500 | | | 0.5800 | 0.0400 | | 0.6200 | 0.0500 | | | 0.6000 | 0.0400 | | 0.6200 | 0.0500 | | | 0.6000 | 0.0500 | | 0.7200 | 0.0900 | -------------------------------------------------------------------------------------------- | Variance | 0.0006 | 0.0001 | | 0.0010 | 0.0001 | | Mean | 0.6025 | 0.0483 | | 0.6408 | 0.0550 | -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- | | Lid closed + no suspend | Lid closed + no suspend | | | | | | Screen off idle (After) | | Screen off idle (Before) | -------------------------------------------------------------------------------------------- | | PkgWatt | CorWatt | | PkgWatt | CorWatt | | | 0.3100 | 0.0700 | | 0.3500 | 0.0700 | | | 0.3100 | 0.0700 | | 0.3200 | 0.0700 | | | 0.3300 | 0.0700 | | 0.4100 | 0.0900 | | | 0.3100 | 0.0600 | | 0.3400 | 0.0700 | | | 0.3300 | 0.0700 | | 0.3200 | 0.0700 | | | 0.3800 | 0.0900 | | 0.3400 | 0.0700 | | | 0.3600 | 0.0700 | | 0.3400 | 0.0700 | | | 0.3100 | 0.0700 | | 0.3300 | 0.0700 | | | 0.3300 | 0.0700 | | 0.3300 | 0.0800 | | | 0.3700 | 0.0800 | | 0.3200 | 0.0700 | | | 0.3500 | 0.0800 | | 0.3400 | 0.0700 | | | 0.2900 | 0.0600 | | 0.4300 | 0.1100 | | | 0.3300 | 0.0700 | | 0.3400 | 0.0700 | | | 0.3300 | 0.0700 | | 0.3200 | 0.0700 | | | 0.3400 | 0.0700 | | 0.3600 | 0.0800 | | | 0.3000 | 0.0700 | | 0.3200 | 0.0700 | | | 0.3400 | 0.0700 | | 0.3300 | 0.0700 | | | 0.3400 | 0.0900 | | 0.3500 | 0.0800 | | | 0.3400 | 0.0700 | | 0.3800 | 0.0800 | -------------------------------------------------------------------------------------------- | Variance | 0.0005 | 0.0001 | | 0.0010 | 0.0001 | | Mean | 0.3028 | 0.0664 | | 0.3174 | 0.0700 | +───────────+─────────────────────────+──────────+───+──────────────────────────+──────────+ Thanks, - Joel
