On Mon, Apr 16, 2018 at 04:58:07PM +0300, Sergey Gorenko wrote: > Some users complain about RNR errors on the target, when heavy > high-priority tasks run on the initiator. After the > investigation, we found out that the receive WRs were exhausted, > because the initiator could not post them on time. > > Receive work reqeusts are posted in chunks of min_posted_rx to > reduce the number of hits to the HCA. The WRs are posted in the > receive completion handler when the number of free receive buffers > reaches min_posted_rx. But on a high-loaded host, receive CQEs > processing can be delayed and all receive WRs will be exhausted. > In this case, the target will get an RNR error. > > To avoid this, we post receive WR, as soon as at least one receive > buffer is freed. This increases the number of hits to the HCA, but > test results show that performance degradation is not significant. > > Performance results running fio (8 jobs, 64 iodepth) using ramdisk > (w/w.o patch): > > bs IOPS(randread) IOPS(randwrite) > ------ --------------- --------------- > 512 329.4K / 340.3K 379.1K / 387.7K > 1K 333.4K / 337.6K 364.2K / 370.0K > 2K 321.1K / 323.5K 334.2K / 337.8K > 4K 300.7K / 302.9K 290.2K / 291.6K > 8K 235.9K / 237.1K 228.2K / 228.8K > 16K 176.3K / 177.0K 126.3K / 125.9K > 32K 115.2K / 115.4K 82.2K / 82.0K > 64K 70.7K / 70.7K 47.8K / 47.6K > 128K 38.5K / 38.6K 25.8K / 25.7K > 256K 20.7K / 20.7K 13.7K / 13.6K > 512K 10.0K / 10.0K 7.0K / 7.0K > > Changes from v0: > - Follow Sagi's suggestion, iser_post_rx_bufs() is refactored. Removed > unnecessary variable and centralized the exit from the function. > The lines above come after "---" and after all Signed-off-by/Reviewed-by. Thanks > Signed-off-by: Sergey Gorenko <sergeygo@xxxxxxxxxxxx> > Signed-off-by: Vladimir Neyelov <vladimirn@xxxxxxxxxxxx> > Reviewed-by: Max Gurtovoy <maxg@xxxxxxxxxxxx> > --- > drivers/infiniband/ulp/iser/iscsi_iser.h | 14 +----- > drivers/infiniband/ulp/iser/iser_initiator.c | 65 +++++++++++++--------------- > drivers/infiniband/ulp/iser/iser_verbs.c | 39 +++++------------ > 3 files changed, 41 insertions(+), 77 deletions(-)
Attachment:
signature.asc
Description: PGP signature
