On Wed, Jul 13, 2022 at 09:32:32PM +0000, Joel Fernandes (Google) wrote: > Hello! > > Please find the next improved version of call_rcu_lazy() attached. The main > difference between the previous versions is that: > - In v2 rcu_barrier is fixed to not hang (I found this to be due to a missing > GP thread wakeup), now I am limiting this wake up only to rcu_barrier() as > requested by Paul. > - Fixed checkpatch and build robot issues. > - Some more changes to 'lazy' parameter passing and consolidation of segcblist > functions. > - more testing via rcutorture and rcuscale. Thank you! What I am going to do is to pull these into an experimental not-for-mainline branch and run the usual set of rcutorture tests. I will then take a look at the patches. > Note that these tests were run on v2 patches, I am expecting similar power > improvements however I've not yet tested power. > > Following are power savings we saw on top of RCU_NOCB_CPU on an Intel platform > in v2. The observation is that due to a 'trickle down' effect of RCU > callbacks, the system is very lightly loaded but constantly running few RCU > callbacks very often. This confuses the power management hardware that the > system is active, when it is in fact idle. > > For example, when ChromeOS screen is off and user is not doing anything on the > system, we can see big power savings. > Before: > Pk%pc10 = 72.13 > PkgWatt = 0.58 > CorWatt = 0.04 > > After: > Pk%pc10 = 81.28 > PkgWatt = 0.41 > CorWatt = 0.03 When you update these numbers, please explain what they all are and evaluate them in the cover letter (or in the relevant patch's commit log). For final submission, please also include some estimate of the variance. For example, CorWatt might be essentially the same both before and after, as in 0.035 and 0.034, or there might be a large difference, as in 0.044 and 0.025. The 81.28 might be constant in all four digits (ha!), or it might vary between (say) 80 and 83. And so on. Based on our earlier emails, my guess is that Pk%pc10 is the percent of time that the system is in a low-power state (bigger is better), PkgWatt is power consumed by the CPU chip (smaller is better), and CorWatt is power consumed by the CPU core (again, smaller is better). It all might sound painfully obvious to you right now, but please have pity on future readers of this series. > Further, when ChromeOS screen is ON but system is idle or lightly loaded, we > can see that the display pipeline is constantly doing RCU callback queuing due > to open/close of file descriptors associated with graphics buffers. This is > attributed to the file_free_rcu() path which this patch series also touches. > > On memory pressure, timeout or queue growing too big, we initiate a flush of of > the bypass lists holding the lazy CBs. > > Similar results can be achieved by increasing jiffies_till_first_fqs, however > that also has the effect of slowing down RCU. Especially I saw huge slow down In the final submission, please quantify "huge slow down". ;-) > of function graph tracer when increasing that. That may be possible to fix via > rcu_expedited=1 boot parameter, however call_rcu_lazy() provides another option > over slowing down ALL call_rcu() globally. Further using jiffies_till_first_fqs > approach will still cause a wake up of the main RCU GP kthread, with this work > we delay even those wakeups. > > One drawback of this series is, if another frequent RCU callback creeps up in > the future, that's not lazy, then that will again hurt the power. However, I > believe identifying and fixing those is a more reasonable approach than slowing > RCU down for the whole system. Like I said earlier, you are the official call_rcu_lazy() whack-a-mole developer. ;-) Thanx, Paul > Disclaimer: I have intentionally not CC'd other subsystem maintainers (like > net, fs) to keep noise low and will CC them in the future after 1 or 2 rounds > of review and agreements. > > Joel Fernandes (Google) (4): > rcu: Introduce call_rcu_lazy() API implementation > rcuscale: Add laziness and kfree tests > fs: Move call_rcu() to call_rcu_lazy() in some paths > rcutorture: Add test code for call_rcu_lazy() > > Vineeth Pillai (1): > rcu: shrinker for lazy rcu > > fs/dcache.c | 4 +- > fs/eventpoll.c | 2 +- > fs/file_table.c | 2 +- > fs/inode.c | 2 +- > include/linux/rcu_segcblist.h | 1 + > include/linux/rcupdate.h | 6 + > kernel/rcu/Kconfig | 8 + > kernel/rcu/rcu.h | 12 + > kernel/rcu/rcu_segcblist.c | 15 +- > kernel/rcu/rcu_segcblist.h | 20 +- > kernel/rcu/rcuscale.c | 74 +++++- > kernel/rcu/rcutorture.c | 60 ++++- > kernel/rcu/tree.c | 132 ++++++---- > kernel/rcu/tree.h | 10 +- > kernel/rcu/tree_nocb.h | 239 ++++++++++++++---- > .../selftests/rcutorture/configs/rcu/TREE11 | 18 ++ > .../rcutorture/configs/rcu/TREE11.boot | 8 + > 17 files changed, 508 insertions(+), 105 deletions(-) > create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/TREE11 > create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/TREE11.boot > > -- > 2.37.0.144.g8ac04bfd2-goog >
