> On Tue, Apr 11, 2023 at 04:58:22PM +0200, Uladzislau Rezki wrote:
> > On Tue, Apr 11, 2023 at 02:42:27PM +0000, Zhang, Qiang1 wrote:
> > > > > Currently, in kfree_rcu_shrink_scan(), the drain_page_cache()
> > > > > is executed before kfree_rcu_monitor() to drain page cache, if
> > > > > the bnode structure's->gp_snap has done, the kvfree_rcu_bulk()
> > > > > will fill the page cache again in kfree_rcu_monitor(), this
> > > > > commit add a check for krcp
> > > > > structure's->backoff_page_cache_fill in put_cached_bnode(), if
> > > > > the krcp structure's->backoff_page_cache_fill is set, prevent page cache growing and disable allocated page in fill_page_cache_func().
> > > > >
> > > > > Signed-off-by: Zqiang <qiang1.zhang@xxxxxxxxx>
> > > > >
> > > > >Much improved! But still some questions below...
> > > > >
> > > > > Thanx, Paul
> > > > >
> > > > > ---
> > > > > kernel/rcu/tree.c | 4 +++-
> > > > > 1 file changed, 3 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index
> > > > > cc34d13be181..9d9d3772cc45 100644
> > > > > --- a/kernel/rcu/tree.c
> > > > > +++ b/kernel/rcu/tree.c
> > > > > @@ -2908,6 +2908,8 @@ static inline bool
> > > > > put_cached_bnode(struct kfree_rcu_cpu *krcp,
> > > > > struct kvfree_rcu_bulk_data *bnode) {
> > > > > + if (atomic_read(&krcp->backoff_page_cache_fill))
> > > > > + return false;
> > > > >
> > > > >This will mean that under low-memory conditions, we will keep
> > > > >zero pages in ->bkvcache. All attempts to put something there will fail.
> > > > >
> > > > >This is probably not an issue for structures containing an
> > > > >rcu_head that are passed to kfree_rcu(p, field), but doesn't
> > > > >this mean that
> > > > >kfree_rcu_mightsleep() unconditionally invokes synchronize_rcu()?
> > > > >This could seriously slow up freeing under low-memory
> > > > >conditions, which might exacerbate the low-memory conditions.
> > > >
> > > > Thanks for mentioning this, I didn't think of this before😊.
> > > >
> > > > >
> > > > >Is this really what we want? Zero cached rather than just fewer cached?
> > > > >
> > > > >
> > > > >
> > > > > // Check the limit.
> > > > > if (krcp->nr_bkv_objs >= rcu_min_cached_objs)
> > > > > return false;
> > > > > @@ -3221,7 +3223,7 @@ static void fill_page_cache_func(struct work_struct *work)
> > > > > int i;
> > > > >
> > > > > nr_pages = atomic_read(&krcp->backoff_page_cache_fill) ?
> > > > > - 1 : rcu_min_cached_objs;
> > > > > + 0 : rcu_min_cached_objs;
> > > > >
> > > > > for (i = 0; i < nr_pages; i++) {
> > > > >
> > > > >I am still confused as to why we start "i" at zero rather than
> > > > >at
> > > > >->nr_bkv_objs. What am I missing here?
> > > >
> > > >
> > > > No, you are right, I missed this place.
> > > >
> > > > --- a/kernel/rcu/tree.c
> > > > +++ b/kernel/rcu/tree.c
> > > > @@ -2908,6 +2908,8 @@ static inline bool
> > > > put_cached_bnode(struct kfree_rcu_cpu *krcp,
> > > > struct kvfree_rcu_bulk_data *bnode) {
> > > > + if (atomic_read(&krcp->backoff_page_cache_fill))
> > > > + return false;
> > > >
> > > >This is broken, unfortunately. If a low memory condition we fill
> > > >fill a cache with at least one page anyway because of we do not
> > > >want to hit a slow path.
> > >
> > > Thanks remind, please ignore my v4 patch, how about the following?
> > >
> > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index
> > > 41daae3239b5..e2e8412e687f 100644
> > > --- a/kernel/rcu/tree.c
> > > +++ b/kernel/rcu/tree.c
> > > @@ -3238,6 +3238,9 @@ static void fill_page_cache_func(struct work_struct *work)
> > > free_page((unsigned long) bnode);
> > > break;
> > > }
> > > +
> > > + if (atomic_read(&krcp->backoff_page_cache_fill))
> > > + break;
> > > }
> > It does not fix an "issue" you are reporting. kvfree_rcu_bulk()
> > function can still fill it back. IMHO, the solution here is to
> > disable cache if a low memory condition and enable back later on.
> >
> > The cache size is controlled by the rcu_min_cached_objs variable. We
> > can set it to 1 and restore it back to original value to make the
> > cache operating as before.
>
> It would be best to use a second variable for this. Users might get
> annoyed if their changes to rcu_min_cached_objs got overwritten once
> things got set back to normal operation.
>
>Agree. So we do not make it visible over sysfs interface for user that we manipulate it.
>
>
The rcu_min_cached_objs is read-only, Users cannot be set rcu_min_cached_objs dynamically.
-r--r--r-- 1 root root 4.0K Apr 12 01:08 rcu_min_cached_objs
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 41daae3239b5..0e9f83562823 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2909,7 +2909,8 @@ put_cached_bnode(struct kfree_rcu_cpu *krcp,
struct kvfree_rcu_bulk_data *bnode)
{
// Check the limit.
- if (krcp->nr_bkv_objs >= rcu_min_cached_objs)
+ if ((atomic_read(&krcp->backoff_page_cache_fill) && krcp->nr_bkv_objs) ||
+ krcp->nr_bkv_objs >= rcu_min_cached_objs)
return false;
llist_add((struct llist_node *) bnode, &krcp->bkvcache);
thoughts?
Thanks
Zqiang
>
>--
>Uladzislau Rezki
