> -----Original Message----- > From: Bob Pearson <rpearsonhpe@xxxxxxxxx> > Sent: Thursday, 16 March 2023 18:15 > To: Bernard Metzler <BMT@xxxxxxxxxxxxxx>; Jason Gunthorpe <jgg@xxxxxxxxxx>; > Zhu Yanjun <zyjzyj2000@xxxxxxxxx>; linux-rdma@xxxxxxxxxxxxxxx > Subject: [EXTERNAL] Re: question about the completion tasklet in the rxe > driver > > On 3/16/23 04:40, Bernard Metzler wrote: > > > > > >> -----Original Message----- > >> From: Bob Pearson <rpearsonhpe@xxxxxxxxx> > >> Sent: Wednesday, 15 March 2023 22:56 > >> To: Jason Gunthorpe <jgg@xxxxxxxxxx>; Zhu Yanjun <zyjzyj2000@xxxxxxxxx>; > >> linux-rdma@xxxxxxxxxxxxxxx; Bernard Metzler <BMT@xxxxxxxxxxxxxx> > >> Subject: [EXTERNAL] question about the completion tasklet in the rxe > driver > >> > >> I have a goal of trying to get rid of all the tasklets in the rxe driver > >> and with the replacement of the > >> three QP tasklets by workqueues the only remaining one is the tasklet > that > >> defers the CQ completion > >> handler. This has been in there since the driver went upstream so the > >> history of why it is there is lost. > >> > >> I notice that the mlx5 driver does have a deferral mechanism for the > >> completion handler but the siw driver > >> does not. I really do not see what advantage, if any, this has for the > rxe > >> driver. Perhaps there is some > >> reason it shouldn't run in hard interrupt context but the CQ tasklet is > a > >> soft interrupt so the completion > >> handler can't sleep anyway. > >> > >> As an experiment I removed the CQ tasklet in the rxe driver and it runs > >> fine. In fact the performance is > >> slightly better with the completion handler called inline rather than > >> deferred to another tasklet. > > > > That is what I would suggest to do. Why would you leave receive > > processing or failing send processing w/o creating the CQE and > > kicking the CQ handler, if you are in a context with > > all information available to build a CQE, signal its availability > > to the consumer and kick a user handler if registered and armed? > > > > Only exception I see: If you process the SQ in post_send() user context > > and a failure results in immediate CQE creation, direct CQ handler > calling > > is not allowed - see Documentation/infiniband/core_locking.rst > > Not sure though, if rxe allows for direct SQ processing out of user > > context. > > > > Cheers, > > Bernard. > > And you did. I am not sure why the mlx5 driver defers the completion > handler call > to a tasklet. I could be that it gets called in a hard interrupt and > completion > handling is deferred to a soft interrupt context. But for rxe the > completion > is always already in a soft interrupt context or a process context. > right, go for it ;) > Bob > > > > >> If we can eliminate this there won't be anymore tasklets in the rxe > driver. > >> > >> Does anyone know why the tasklet was put in in the first place? > >> > >> Thanks, > >> > >> Bob
