On Mon, Aug 12, 2019 at 07:29:17PM +0900, Byungchul Park wrote:
[snip]
> > diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
> > index 8e727f57d814..383f2481750f 100644
> > --- a/include/linux/rcutiny.h
> > +++ b/include/linux/rcutiny.h
> > @@ -39,6 +39,11 @@ static inline void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
> > call_rcu(head, func);
> > }
> >
> > +static inline void kfree_call_rcu_nobatch(struct rcu_head *head, rcu_callback_t func)
> > +{
> > + call_rcu(head, func);
> > +}
> > +
> > void rcu_qs(void);
> >
> > static inline void rcu_softirq_qs(void)
> > diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
> > index 735601ac27d3..7e38b39ec634 100644
> > --- a/include/linux/rcutree.h
> > +++ b/include/linux/rcutree.h
> > @@ -34,6 +34,7 @@ static inline void rcu_virt_note_context_switch(int cpu)
> >
> > void synchronize_rcu_expedited(void);
> > void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func);
> > +void kfree_call_rcu_nobatch(struct rcu_head *head, rcu_callback_t func);
> >
> > void rcu_barrier(void);
> > bool rcu_eqs_special_set(int cpu);
> > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > index a14e5fbbea46..102a5f606d78 100644
> > --- a/kernel/rcu/tree.c
> > +++ b/kernel/rcu/tree.c
> > @@ -2593,17 +2593,204 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
> > }
> > EXPORT_SYMBOL_GPL(call_rcu);
> >
> > +
> > +/* Maximum number of jiffies to wait before draining a batch. */
> > +#define KFREE_DRAIN_JIFFIES (HZ / 50)
>
> JFYI, I also can see oom with a larger value of this. I hope this magic
> value works well for all kind of systems.
It seems to work well in my testing. I am glad you could not perceive OOMs at
this value, either.
> > - * Queue an RCU callback for lazy invocation after a grace period.
> > - * This will likely be later named something like "call_rcu_lazy()",
> > - * but this change will require some way of tagging the lazy RCU
> > - * callbacks in the list of pending callbacks. Until then, this
> > - * function may only be called from __kfree_rcu().
> > + * Maximum number of kfree(s) to batch, if this limit is hit then the batch of
> > + * kfree(s) is queued for freeing after a grace period, right away.
> > */
> > -void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
> > +struct kfree_rcu_cpu {
> > + /* The rcu_work node for queuing work with queue_rcu_work(). The work
> > + * is done after a grace period.
> > + */
> > + struct rcu_work rcu_work;
> > +
> > + /* The list of objects being queued in a batch but are not yet
> > + * scheduled to be freed.
> > + */
> > + struct rcu_head *head;
> > +
> > + /* The list of objects that have now left ->head and are queued for
> > + * freeing after a grace period.
> > + */
> > + struct rcu_head *head_free;
> > +
> > + /* Protect concurrent access to this structure. */
> > + spinlock_t lock;
> > +
> > + /* The delayed work that flushes ->head to ->head_free incase ->head
> > + * did not reach a length of KFREE_MAX_BATCH within KFREE_DRAIN_JIFFIES.
> > + * In case flushing cannot be done if RCU is busy, then ->head just
> > + * continues to grow beyond KFREE_MAX_BATCH and we retry flushing later.
>
> Minor one. We don't use KFREE_MAX_BATCH anymore.
Sorry for leaving these KFREE_MAX_BATCH comments stale, I cleaned up many of
them already but some where still left behind. I will fix these in the v3.
thanks for review!
- Joel
[snip]
