On 2020-05-25 12:15, Bart Van Assche wrote:
> On 2020-05-25 10:51, Leon Romanovsky wrote:
>> On Mon, May 25, 2020 at 10:22:12AM -0700, Bart Van Assche wrote:
>>> The ib_srpt driver limits max_send_sge to 16. Since that is a workaround
>>> for an mlx4 bug that has been fixed, increase max_send_sge. For mlx4, do
>>> not use the value advertised by the driver (32) since that causes QP's
>>> to transition to the error status.
>>
>> How are you avoiding mlx4 bug in this patch?
>> Isn't "attrs->max_send_sge" come from driver as is?
>
> Hi Leon,
>
> Development of this patch started considerable time ago - before the
> ib_srpt driver was converted to the RDMA R/W API. Before that conversion
> the ib_srpt driver was using attrs->max_send_sge limit for both RDMA
> writes and RDMA reads, which is wrong. Hence the need for the
> "max_sge_delta" parameter (max_send_sge = 32 and max_sge_rd = 30 for
> mlx4). The following code from drivers/infiniband/core/rw.c selects the
> proper limit:
>
> u32 max_sge = dir == DMA_TO_DEVICE ? qp->max_write_sge :
> qp->max_read_sge;
>
> The following code in drivers/infiniband/core/verbs.c sets these limits:
>
> qp->max_write_sge = qp_init_attr->cap.max_send_sge;
> qp->max_read_sge = min_t(u32, qp_init_attr->cap.max_send_sge,
> device->attrs.max_sge_rd);
The commit message should be shortened to the following: "The ib_srpt
driver limits max_send_sge to 16. Since that is a workaround
for an mlx4 bug that has been fixed, increase max_send_sge. See also
commit f95ccffc715b ("IB/mlx4: Use 4K pages for kernel QP's WQE buffer")."
Bart.
