Hi Guoqing, On Fri, Apr 29, 2022 at 10:26 AM Guoqing Jiang <guoqing.jiang@xxxxxxxxx> wrote: > > Hi Haris, > > On 4/20/22 6:28 PM, Haris Iqbal wrote: > > Hello, > > > > We are facing some issues with the rxe driver in v5.14 (tested with 5.14.21) > > > > After mapping a single RNBD device with 2 rxe interfaces, and with the > > below fio config, > > > > [global] > > description=Emulation of Storage Server Access Pattern > > bssplit=512/20:1k/16:2k/9:4k/12:8k/19:16k/10:32k/8:64k/4 > > fadvise_hint=0 > > rw=randrw:2 > > direct=1 > > random_distribution=zipf:1.2 > > time_based=1 > > runtime=60 > > ramp_time=1 > > ioengine=libaio > > iodepth=128 > > iodepth_batch_submit=128 > > iodepth_batch_complete_min=1 > > iodepth_batch_complete_max=128 > > numjobs=1 > > group_reporting > > > > [job1] > > filename=/dev/rnbd0 > > > > > > We observe the following error, > > > > [Fri Mar 25 19:08:03 2022] rtrs_client L353: <blya>: Failed > > IB_WR_LOCAL_INV: WR flushed > > [Fri Mar 25 19:08:03 2022] rtrs_client L333: <blya>: Failed > > IB_WR_REG_MR: WR flushed > > [Fri Mar 25 19:08:03 2022] rtrs_client L333: <blya>: Failed > > IB_WR_REG_MR: WR flushed > > [Fri Mar 25 19:08:34 2022] rtrs_client L353: <blya>: Failed > > IB_WR_LOCAL_INV: WR flushed > > [Fri Mar 25 19:08:34 2022] rtrs_client L353: <blya>:*Failed IB_WR_LOCAL_INV: WR flushed* > > I got similar message but I am not certain it is the same one, pls see > the previous report, > > https://lore.kernel.org/linux-rdma/20220210073655.42281-1-guoqing.jiang@xxxxxxxxx/ I went through the emails, and the error looks similar but I am not sure if its related. The change to write_lock_bh was done after v5.14 I think, and the code where I am seeing this error still has irq versions of those lock (e.g. write_lock_irqsave). I don't have much knowledge of rxe code, hence I cannot really say for sure. Thanks for the response. > > Thanks, > Guoqing
