On Fri, Oct 11, 2019 at 10:10 PM John Garry <john.garry@xxxxxxxxxx> wrote: > > On 11/10/2019 12:55, Ming Lei wrote: > > On Fri, Oct 11, 2019 at 4:54 PM John Garry <john.garry@xxxxxxxxxx> wrote: > >> > >> On 10/10/2019 12:21, John Garry wrote: > >>> > >>>> > >>>> As discussed before, tags of hisilicon V3 is HBA wide. If you switch > >>>> to real hw queue, each hw queue has to own its independent tags. > >>>> However, that isn't supported by V3 hardware. > >>> > >>> I am generating the tag internally in the driver now, so that hostwide > >>> tags issue should not be an issue. > >>> > >>> And, to be clear, I am not paying too much attention to performance, but > >>> rather just hotplugging while running IO. > >>> > >>> An update on testing: > >>> I did some scripted overnight testing. The script essentially loops like > >>> this: > >>> - online all CPUS > >>> - run fio binded on a limited bunch of CPUs to cover a hctx mask for 1 > >>> minute > >>> - offline those CPUs > >>> - wait 1 minute (> SCSI or NVMe timeout) > >>> - and repeat > >>> > >>> SCSI is actually quite stable, but NVMe isn't. For NVMe I am finding > >>> some fio processes never dying with IOPS @ 0. I don't see any NVMe > >>> timeout reported. Did you do any NVMe testing of this sort? > >>> > >> > >> Yeah, so for NVMe, I see some sort of regression, like this: > >> Jobs: 1 (f=1): [_R] [0.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta > >> 1158037877d:17h:18m:22s] > > > > I can reproduce this issue, and looks there are requests in ->dispatch. > > OK, that may match with what I see: > - the problem occuring coincides with this callpath with > BLK_MQ_S_INTERNAL_STOPPED set: Good catch, these requests should have been re-submitted in blk_mq_hctx_notify_dead() too. Will do it in V4. Thanks, Ming Lei
