On Tue, Oct 20, 2020 at 05:04:29PM +0800, Chao Leng wrote:
>
>
> On 2020/10/20 16:53, Ming Lei wrote:
> > During controller's CONNECTING state, admin/fabric/connect requests
> > are submitted for recovery controller, and we allow to abort this request
> > directly in time out handler for not blocking setup procedure.
> >
> > So timout vs. normal completion race exists on these requests since
> > admin/fabirc/connect queues won't be shutdown before handling timeout
> > during CONNECTING state.
> >
> > Add atomic completion for requests from connect/fabric/admin queue for
> > avoiding the race.
> >
> > CC: Chao Leng <lengchao@xxxxxxxxxx>
> > Cc: Sagi Grimberg <sagi@xxxxxxxxxxx>
> > Reported-by: Yi Zhang <yi.zhang@xxxxxxxxxx>
> > Tested-by: Yi Zhang <yi.zhang@xxxxxxxxxx>
> > Signed-off-by: Ming Lei <ming.lei@xxxxxxxxxx>
> > ---
> > drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
> > 1 file changed, 37 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > index d6a3e1487354..7e85bd4a8d1b 100644
> > --- a/drivers/nvme/host/tcp.c
> > +++ b/drivers/nvme/host/tcp.c
> > @@ -30,6 +30,8 @@ static int so_priority;
> > module_param(so_priority, int, 0644);
> > MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
> > +#define REQ_STATE_COMPLETE 0
> > +
> > enum nvme_tcp_send_state {
> > NVME_TCP_SEND_CMD_PDU = 0,
> > NVME_TCP_SEND_H2C_PDU,
> > @@ -56,6 +58,8 @@ struct nvme_tcp_request {
> > size_t offset;
> > size_t data_sent;
> > enum nvme_tcp_send_state state;
> > +
> > + unsigned long comp_state;
> I do not think adding state is a good idea.
> It is similar to rq->state.
> In the teardown process, after quiesced queues delete the timer and
> cancel the timeout work maybe a better option.
> I will send the patch later.
> The patch is already tested with roce more than one week.
Actually there isn't race between timeout and teardown, and patch 1 and patch
2 are enough to fix the issue reported by Yi.
It is just that rq->state is updated to IDLE in its. complete(), so
either one of code paths may think that this rq isn't completed, and
patch 2 has addressed this issue.
In short, teardown lock is enough to cover the race.
Thanks,
Ming
