Re: [PATCH 1/1] Revert "RDMA/rxe: Add workqueue support for rxe tasks"

[Bart Van Assche <bvanassche@xxxxxxx>]

On 10/3/23 20:41, Zhu Yanjun wrote:
> 在 2023/9/27 4:24, Bart Van Assche 写道:
> > diff --git a/drivers/infiniband/sw/rxe/rxe_task.c 
> > b/drivers/infiniband/sw/rxe/rxe_task.c
> > index 1501120d4f52..6cd5d5a7a316 100644
> > --- a/drivers/infiniband/sw/rxe/rxe_task.c
> > +++ b/drivers/infiniband/sw/rxe/rxe_task.c
> > @@ -10,7 +10,7 @@ static struct workqueue_struct *rxe_wq;
> >
> >   int rxe_alloc_wq(void)
> >   {
> > -       rxe_wq = alloc_workqueue("rxe_wq", WQ_UNBOUND, WQ_MAX_ACTIVE);
> > +       rxe_wq = alloc_workqueue("rxe_wq", WQ_UNBOUND, 1);
> >          if (!rxe_wq)
> >                  return -ENOMEM;
>
> Hi, Bart
>
> With the above commit, I still found a similar problem. But the problem 
> occurs very rarely. With the following, to now, the problem does not occur.
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_task.c 
> b/drivers/infiniband/sw/rxe/rxe_task.c
> index 1501120d4f52..3189c3705295 100644
> --- a/drivers/infiniband/sw/rxe/rxe_task.c
> +++ b/drivers/infiniband/sw/rxe/rxe_task.c
> @@ -10,7 +10,7 @@ static struct workqueue_struct *rxe_wq;
>
>   int rxe_alloc_wq(void)
>   {
> -       rxe_wq = alloc_workqueue("rxe_wq", WQ_UNBOUND, WQ_MAX_ACTIVE);
> +       rxe_wq = alloc_workqueue("rxe_wq", WQ_HIGHPRI | WQ_UNBOUND, 1);
>          if (!rxe_wq)
>                  return -ENOMEM;
>
>
> And with the tasklet, this problem also does not occur.
>
> With "alloc_workqueue("rxe_wq", WQ_HIGHPRI | WQ_UNBOUND, 1);", an 
> ordered workqueue with high priority is allocated.
>
> To the same number of work item, the ordered workqueue has the same 
> runing time with the tasklet. But the tasklet is based on softirq. Its 
> overhead on scheduling is less than workqueue. So in theory, tasklet's 
> performance should be better than the ordered workqueue.

Hi Zhu,

Thank you for having reported this. I'm OK with integrating the above 
change in my patch. However, code changes must be motivated. Do you 
perhaps have an explanation of why WQ_HIGHPRI makes the issue disappear 
that you observed?

Regarding tasklets: tasklets are a nightmare from the point of view of
the Linux kernel scheduler. As an example, these may make it impossible
for the scheduler to schedule real-time threads in time.

Thanks,

Bart.