Thanks. Good catch. Should be &.
-----Original Message-----
From: Zhu Yanjun <zyjzyj2000@xxxxxxxxx>
Sent: Wednesday, May 26, 2021 12:54 AM
To: Bob Pearson <rpearsonhpe@xxxxxxxxx>
Cc: Jason Gunthorpe <jgg@xxxxxxxxxx>; zyj2020@xxxxxxxxx; RDMA mailing list <linux-rdma@xxxxxxxxxxxxxxx>
Subject: Re: [PATCH for-next] RDMA/rxe: Fix memory ordering problem for resize cq
On Wed, May 26, 2021 at 6:27 AM Bob Pearson <rpearsonhpe@xxxxxxxxx> wrote:
>
> The rxe driver has recently begun exhibiting failures in the python
> tests that are due to stale values read from the completion queue
> producer or consumer indices. Unlike the other loads of these shared
> indices those in queue_count() were not protected by smp_load_acquire().
>
> This patch replaces loads by smp_load_acquire() in queue_count().
> The observed errors no longer occur.
>
> Reported-by: Zhu Yanjun <zyj2020@xxxxxxxxx>
> Fixes: d21a1240f516 ("RDMA/rxe: Use acquire/release for memory
> ordering")
> Signed-off-by: Bob Pearson <rpearsonhpe@xxxxxxxxx>
> ---
> drivers/infiniband/sw/rxe/rxe_queue.h | 18 ++++++++++++++++--
> 1 file changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_queue.h
> b/drivers/infiniband/sw/rxe/rxe_queue.h
> index 2902ca7b288c..5cb142282fa6 100644
> --- a/drivers/infiniband/sw/rxe/rxe_queue.h
> +++ b/drivers/infiniband/sw/rxe/rxe_queue.h
> @@ -161,8 +161,22 @@ static inline unsigned int index_from_addr(const
> struct rxe_queue *q,
>
> static inline unsigned int queue_count(const struct rxe_queue *q) {
> - return (q->buf->producer_index - q->buf->consumer_index)
> - & q->index_mask;
> + u32 prod;
> + u32 cons;
> + u32 count;
> +
> + /* make sure all changes to queue complete before
> + * changing producer index
> + */
> + prod = smp_load_acquire(&q->buf->producer_index);
> +
> + /* make sure all changes to queue complete before
> + * changing consumer index
> + */
> + cons = smp_load_acquire(&q->buf->consumer_index);
> + count = (prod - cons) % q->index_mask;
% is different from &. Not sure it is correct to use % instead of & in the original source code.
Zhu Yanjun
> +
> + return count;
> }
>
> static inline void *queue_head(struct rxe_queue *q)
> --
> 2.30.2
>
