On 18/02/2021 18:23, Jason Gunthorpe wrote: > On Thu, Feb 18, 2021 at 05:52:16PM +0200, Gal Pressman wrote: >> On 18/02/2021 14:53, Jason Gunthorpe wrote: >>> On Thu, Feb 18, 2021 at 11:13:43AM +0200, Gal Pressman wrote: >>>> I'm a bit confused about the meaning of the ibv_req_notify_cq() verb: >>>> "Upon the addition of a new CQ entry (CQE) to cq, a completion event will be >>>> added to the completion channel associated with the CQ." >>>> >>>> What is considered a new CQE in this case? >>>> The next CQE from the user's perspective, i.e. any new CQE that wasn't consumed >>>> by the user's poll cq? >>>> Or any new CQE from the device's perspective? >>> >>> new CQE from the device perspective. >>> >>>> For example, if at the time of ibv_req_notify_cq() call the CQ has received 100 >>>> completions, but the user hasn't polled his CQ yet, when should he be notified? >>>> On the 101 completion or immediately (since there are completions waiting on the >>>> CQ)? >>> >>> 101 completion >>> >>> It is only meaningful to call it when the CQ is empty. >> >> Thanks, so there's an inherent race between the user's CQ poll and the next arm? > > I think the specs or man pages talk about this, the application has to > observe empty, do arm, then poll again then sleep on the cq if empty. > >> Do you know what's the purpose of the consumer index in the arm doorbell that's >> implemented by many providers? > > The consumer index is needed by HW to prevent CQ overflow, presumably > the drivers push to reduce the cases where the HW has to read it from > PCI Thanks, that makes sense. I found the following sentence in CX PRM: "If new CQEs are posted to the CQ after the reporting of a completion event and these CQEs are not yet consumed, then an event will be generated immediately after the request for notification is executed." Doesn't that contradict the expected behavior?
