On 05/03/2016 02:33 AM, Christoph Hellwig wrote:
> On Fri, Apr 22, 2016 at 02:16:31PM -0700, Bart Van Assche wrote:
>> If both max_sectors and the queue_depth are high enough it can
>> happen that the MR pool is depleted temporarily. This causes
>> the SRP initiator to report mapping failures. Although the SRP
>> initiator recovers from such mapping failures, prevent that
>> this can happen by limiting max_sectors.
>
> FYI, even with this patch I see tons of errors like:
>
> [ 2237.161106] scsi host7: ib_srp: Failed to map data (-12)
That's unintended. I can reproduce this and will analyze this further.
>> + /*
>> + * FR and FMR can only map one HCA page per entry. If the
>> + * start address is not aligned on a HCA page boundary two
>> + * entries will be used for the head and the tail although
>> + * these two entries combined contain at most one HCA page of
>> + * data. Hence the "- 1" in the calculation below.
>> + */
>> + max_max_sectors = (srp_dev->max_pages_per_mr - 1) <<
>> + (ilog2(srp_dev->mr_page_size) - 9);
>> + if (target->scsi_host->max_sectors > max_max_sectors) {
>> + shost_printk(KERN_WARNING, target->scsi_host,
>> + PFX "Reducing max_sectors from %d to %d\n",
>> + target->scsi_host->max_sectors,
>> + max_max_sectors);
>> + target->scsi_host->max_sectors = max_max_sectors;
>> + }
>
> I don't think there is any good reason to printk a warning here -
> limited hardware is a totally normal thing. E.g. if we merge
> your RDMA/CM support and someone runs SRP on chelsio hardware they'd
> probably hit this all the time..
Are you sure? What I see in the v4.6-rc6 tree seems to indicate that Chelsio
hardware supports large page lists:
$ git grep -nHw T[34]_MAX_MR_SIZE
drivers/infiniband/hw/cxgb3/cxio_hal.h:58:#define T3_MAX_MR_SIZE 0x100000000ULL
drivers/infiniband/hw/cxgb3/iwch.c:125: rnicp->attr.max_mr_size = T3_MAX_MR_SIZE;
drivers/infiniband/hw/cxgb4/provider.c:328: props->max_mr_size = T4_MAX_MR_SIZE;
drivers/infiniband/hw/cxgb4/t4.h:41:#define T4_MAX_MR_SIZE (~0ULL)
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
