Acked. Will fix.
-----Original Message-----
From: Jason Gunthorpe
Sent: Tuesday, December 18, 2018 11:40 AM
To: Leon Romanovsky <leon@xxxxxxxxxx>
Cc: Doug Ledford <dledford@xxxxxxxxxx>; Huy Nguyen <huyn@xxxxxxxxxxxx>; RDMA mailing list <linux-rdma@xxxxxxxxxxxxxxx>; Daniel Jurgens <danielj@xxxxxxxxxxxx>; Haggai Eran <haggaie@xxxxxxxxxxxx>; Leon Romanovsky <leonro@xxxxxxxxxxxx>; Moni Shoua <monis@xxxxxxxxxxxx>
Subject: Re: [PATCH rdma-next] IB/mlx5: Fix long EEH recover time with NVMe offloads
On Tue, Dec 18, 2018 at 02:15:56PM +0200, Leon Romanovsky wrote:
> From: Huy Nguyen <huyn@xxxxxxxxxxxx>
>
> On NVMe offloads connection with many IO queues, EEH takes long time
> to recover. The culprit is the synchronize_srcu in the destroy_mkey.
> Solution is to use synchronize_srcu only for ODP mkey.
>
> Fixes: b4cfe447d47b ("IB/mlx5: Implement on demand paging by adding
> support for MMU notifiers")
> Signed-off-by: Huy Nguyen <huyn@xxxxxxxxxxxx>
> Reviewed-by: Daniel Jurgens <danielj@xxxxxxxxxxxx>
> Signed-off-by: Leon Romanovsky <leonro@xxxxxxxxxxxx>
> drivers/infiniband/hw/mlx5/mr.c | 19 ++++++++++++++++---
> 1 file changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/hw/mlx5/mr.c
> b/drivers/infiniband/hw/mlx5/mr.c index d45a9163c198..0f01059724e9
> 100644
> +++ b/drivers/infiniband/hw/mlx5/mr.c
> @@ -73,7 +73,8 @@ static int destroy_mkey(struct mlx5_ib_dev *dev,
> struct mlx5_ib_mr *mr)
>
> #ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> /* Wait until all page fault handlers using the mr complete. */
> - synchronize_srcu(&dev->mr_srcu);
> + if (mr->umem && mr->umem->is_odp)
> + synchronize_srcu(&dev->mr_srcu);
> #endif
We don't need any of these gross #ifdef's..
Add a
static inline bool is_odp_mr(struct mlx5_ib_mr *mr) {
return IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING) && mr->umem && mr->umem->is_odp; }
And use it in place of all the is_odp tests.
Since it returns the constant false if ODP is disabled the compiler will drop all the branches naturally without the horrible #ifdef mess.
Someone should send a cleanup patch for ODP to get rid of more #ifdefs.
Jason
