> > > Nice, I am happy you worked on it. A comment bellow: > > Below what i have came up with to switch for polling APIs: > > > > From 799ce1653d159ef3d35f34a284f738c2c267c75f Mon Sep 17 00:00:00 2001 > > From: "Uladzislau Rezki (Sony)" <urezki@xxxxxxxxx> > > Date: Wed, 2 Nov 2022 19:26:27 +0100 > > Subject: [PATCH 1/1] rcu: kvfree_rcu: Reduce a memory footptint by using > > polling APIs > > > > Total time taken by all kfree'ers: 6564718459 ns, loops: 10000, batches: > > 1110, memory footprint: 5057MB > > Total time taken by all kfree'ers: 8431051895 ns, loops: 10000, batches: > > 1109, memory footprint: 2749MB > > Total time taken by all kfree'ers: 9477830789 ns, loops: 10000, batches: > > 1158, memory footprint: 2934MB > > Total time taken by all kfree'ers: 9950211144 ns, loops: 10000, batches: > > 981, memory footprint: 2704MB > > > > with a patch: > > > > > > Total time taken by all kfree'ers: 7712110118 ns, loops: 10000, batches: > > 1660, memory footprint: 91MB > > Total time taken by all kfree'ers: 7002403664 ns, loops: 10000, batches: > > 1482, memory footprint: 86MB > > Total time taken by all kfree'ers: 7842282319 ns, loops: 10000, batches: > > 1738, memory footprint: 86MB > > Total time taken by all kfree'ers: 7230161977 ns, loops: 10000, batches: > > 1542, memory footprint: 72MB > > > > Tested with NOCB option, all offloading CPUs: > > > > kvm.sh --memory 10G --torture rcuscale --allcpus --duration 1 \ > > --kconfig CONFIG_NR_CPUS=64 \ > > --kconfig CONFIG_RCU_NOCB_CPU=y \ > > --kconfig CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y \ > > --bootargs "rcuscale.kfree_rcu_test=1 rcuscale.kfree_nthreads=16 \ > > rcuscale.holdoff=20 rcuscale.kfree_loops=10000 > > torture.disable_onoff_at_boot" --trust-make > > > > According to data there is a big gain in memory footprint with a patch. > > It is because of call_rcu() and call_rcu_flush() take more effort and > > time to queue a callback and then wait for a gp. > > > > > > With polling API: > > a) we do not need to queue any callback; > > b) we might not even need wait for a GP completion. > > > > Wow, I did not expect an extra grace period or 2 to cause so much memory > unreclaimed. > > Does this kernel have the 5 second timeout change? > Yes. But it does not matter. Why? Because 5 second time out happens only when a system lightly loaded. It can be 10 or 100 seconds. Once a flood is detected we start to offload back-logged memory. > > If yes, then I am confused why extra grace period matters for memory. My > goal with the idea was to avoid pointless RCU work when we don’t need it. > But I can’t complaint. > This patch does the same. It avoids of using any call_rcu() because we do not need it. I mean extra layer. Why it matters for the memory. I explained it in previous answer. But it is because of extra layer. Since it is a flood scenario such delays can lead to higher memory footprint. The test easily detects it. > > Or is it that this test floods the pointer page so we do not wait full 5 > seconds, and benefit when we reclaim sooner with polled API, than waiting > for grace periods? > Right. Reclaim sooner with polling API. -- Uladzislau Rezki
