On 3/16/23 04:40, Bernard Metzler wrote: > > >> -----Original Message----- >> From: Bob Pearson <rpearsonhpe@xxxxxxxxx> >> Sent: Wednesday, 15 March 2023 22:56 >> To: Jason Gunthorpe <jgg@xxxxxxxxxx>; Zhu Yanjun <zyjzyj2000@xxxxxxxxx>; >> linux-rdma@xxxxxxxxxxxxxxx; Bernard Metzler <BMT@xxxxxxxxxxxxxx> >> Subject: [EXTERNAL] question about the completion tasklet in the rxe driver >> >> I have a goal of trying to get rid of all the tasklets in the rxe driver >> and with the replacement of the >> three QP tasklets by workqueues the only remaining one is the tasklet that >> defers the CQ completion >> handler. This has been in there since the driver went upstream so the >> history of why it is there is lost. >> >> I notice that the mlx5 driver does have a deferral mechanism for the >> completion handler but the siw driver >> does not. I really do not see what advantage, if any, this has for the rxe >> driver. Perhaps there is some >> reason it shouldn't run in hard interrupt context but the CQ tasklet is a >> soft interrupt so the completion >> handler can't sleep anyway. >> >> As an experiment I removed the CQ tasklet in the rxe driver and it runs >> fine. In fact the performance is >> slightly better with the completion handler called inline rather than >> deferred to another tasklet. > > That is what I would suggest to do. Why would you leave receive > processing or failing send processing w/o creating the CQE and > kicking the CQ handler, if you are in a context with > all information available to build a CQE, signal its availability > to the consumer and kick a user handler if registered and armed? > > Only exception I see: If you process the SQ in post_send() user context > and a failure results in immediate CQE creation, direct CQ handler calling > is not allowed - see Documentation/infiniband/core_locking.rst > Not sure though, if rxe allows for direct SQ processing out of user > context. > > Cheers, > Bernard. And you did. I am not sure why the mlx5 driver defers the completion handler call to a tasklet. I could be that it gets called in a hard interrupt and completion handling is deferred to a soft interrupt context. But for rxe the completion is always already in a soft interrupt context or a process context. Bob > >> If we can eliminate this there won't be anymore tasklets in the rxe driver. >> >> Does anyone know why the tasklet was put in in the first place? >> >> Thanks, >> >> Bob
