On 2020-05-25 10:51, Leon Romanovsky wrote:
> On Mon, May 25, 2020 at 10:22:12AM -0700, Bart Van Assche wrote:
>> The ib_srpt driver limits max_send_sge to 16. Since that is a workaround
>> for an mlx4 bug that has been fixed, increase max_send_sge. For mlx4, do
>> not use the value advertised by the driver (32) since that causes QP's
>> to transition to the error status.
>
> How are you avoiding mlx4 bug in this patch?
> Isn't "attrs->max_send_sge" come from driver as is?
Hi Leon,
Development of this patch started considerable time ago - before the
ib_srpt driver was converted to the RDMA R/W API. Before that conversion
the ib_srpt driver was using attrs->max_send_sge limit for both RDMA
writes and RDMA reads, which is wrong. Hence the need for the
"max_sge_delta" parameter (max_send_sge = 32 and max_sge_rd = 30 for
mlx4). The following code from drivers/infiniband/core/rw.c selects the
proper limit:
u32 max_sge = dir == DMA_TO_DEVICE ? qp->max_write_sge :
qp->max_read_sge;
The following code in drivers/infiniband/core/verbs.c sets these limits:
qp->max_write_sge = qp_init_attr->cap.max_send_sge;
qp->max_read_sge = min_t(u32, qp_init_attr->cap.max_send_sge,
device->attrs.max_sge_rd);
Bart.
