Under circustances with high load, the driver is running out of
async receive buffers which may result in one of the following
messages:
0:6401 RQE Error x13, posted 226 err_cnt 0: 925c6050 925c604e 925c5d54
or
0:2885 Port Status Event: port status reg 0x81800000,
port smphr reg 0xc000, error 1=0x52004a01, error 2=0x0
The driver is waiting for full io completion before returning receive
buffers to the adapter. There is no need for such a relationship.
Whenever a new command is received from the wire, the driver will
have two contexts - an io context (ctxp) and a receive buffer context.
In current code, the receive buffer context stays 1:1 with the io
and won't be reposted to the hardware until the io completes. There
is no need for such a relationship.
Change the driver so that up on successful handing of the command
to the transport, where the transport has copied what it needed thus
the buffer is returned to the driver, have the driver immediately
repost the buffer to the hardware. If the command cannot be
successfully handed to the transport as transport resources are
temporarily busy, have the driver allocate a new and separate
receive buffer and post it to the hardware so that hardware can
continue while the command is queued for the transport.
When an io is complete, the transport returns the io context to
the driver, and the driver may be waiting for more contexts, thus
immediately reuse the io context. In this path, there was a buffer
posted when the receive buffer was queued waiting for an io context
so a replacement is not needed in the new code additions. Thus,
exempt this the context reuse case from the buffer reposting.
Signed-off-by: Dick Kennedy <dick.kennedy@xxxxxxxxxxxx>
Signed-off-by: James Smart <jsmart2021@xxxxxxxxx>
---
drivers/scsi/lpfc/lpfc_nvmet.c | 25 ++++++++++++++++++++++++-
1 file changed, 24 insertions(+), 1 deletion(-)
diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c
index caf641ee6210..e7ce437007b1 100644
--- a/drivers/scsi/lpfc/lpfc_nvmet.c
+++ b/drivers/scsi/lpfc/lpfc_nvmet.c
@@ -1841,7 +1841,7 @@ lpfc_nvmet_process_rcv_fcp_req(struct lpfc_nvmet_ctxbuf *ctx_buf)
struct lpfc_hba *phba = ctxp->phba;
struct rqb_dmabuf *nvmebuf = ctxp->rqb_buffer;
struct lpfc_nvmet_tgtport *tgtp;
- uint32_t *payload;
+ uint32_t *payload, qno;
uint32_t rc;
unsigned long iflags;
@@ -1874,6 +1874,15 @@ lpfc_nvmet_process_rcv_fcp_req(struct lpfc_nvmet_ctxbuf *ctx_buf)
/* Process FCP command */
if (rc == 0) {
atomic_inc(&tgtp->rcv_fcp_cmd_out);
+ spin_lock_irqsave(&ctxp->ctxlock, iflags);
+ if ((ctxp->flag & LPFC_NVMET_CTX_REUSE_WQ) ||
+ (nvmebuf != ctxp->rqb_buffer)) {
+ spin_unlock_irqrestore(&ctxp->ctxlock, iflags);
+ return;
+ }
+ ctxp->rqb_buffer = NULL;
+ spin_unlock_irqrestore(&ctxp->ctxlock, iflags);
+ lpfc_rq_buf_free(phba, &nvmebuf->hbuf); /* repost */
return;
}
@@ -1884,6 +1893,20 @@ lpfc_nvmet_process_rcv_fcp_req(struct lpfc_nvmet_ctxbuf *ctx_buf)
ctxp->oxid, ctxp->size, ctxp->sid);
atomic_inc(&tgtp->rcv_fcp_cmd_out);
atomic_inc(&tgtp->defer_fod);
+ spin_lock_irqsave(&ctxp->ctxlock, iflags);
+ if (ctxp->flag & LPFC_NVMET_CTX_REUSE_WQ) {
+ spin_unlock_irqrestore(&ctxp->ctxlock, iflags);
+ return;
+ }
+ spin_unlock_irqrestore(&ctxp->ctxlock, iflags);
+ /*
+ * Post a replacement DMA buffer to RQ and defer
+ * freeing rcv buffer till .defer_rcv callback
+ */
+ qno = nvmebuf->idx;
+ lpfc_post_rq_buffer(
+ phba, phba->sli4_hba.nvmet_mrq_hdr[qno],
+ phba->sli4_hba.nvmet_mrq_data[qno], 1, qno);
return;
}
atomic_inc(&tgtp->rcv_fcp_cmd_drop);
--
2.13.7
