On 6/27/23 14:21, Sagi Grimberg wrote: > >> Hello >> >> I found this failure on the latest linux tree, and it cannot be >> reproduced on v6.4, >> it should be one regression recently merged to linux tree after v6.4. >> I check the commit recently merged after v6.4, and found below commit >> touched the related code, not sure if it was introduced by this >> commit. >> >> commit 959ffef13bac792e4e2e3321d6e2bd2b00c0f5f9 >> Author: Chaitanya Kulkarni <kch@xxxxxxxxxx> >> Date: Thu Jun 1 23:47:42 2023 -0700 >> >> nvme-fabrics: open code __nvmf_host_find() > > That just moved code, no functional change, > most likely the below was the offender: > ae8bd606e09b ("nvme-fabrics: prevent overriding of existing host") For nvme-6.5 blktests are passing with nvme_trtype "loop" and "tcp" see [1] with following HEAD :- commit 99160af413b4ff1c3b4741e8a7583f8e7197f201 (origin/nvme-6.5) Author: Sagi Grimberg <sagi@xxxxxxxxxxx> Date: Tue Jun 20 16:07:36 2023 +0300 nvme-mpath: fix I/O failure with EAGAIN when failing over I/O Also, didn't find the following error messsage in dmesg. blktests (master) # dmesg | grep "found same hostid" blktests (master) # confused exactly how blktests are passing on nvme-6.5 branch ? I'll try linux v6.4 next and update you soon .. -ck [1] blktests (master) # ./test-nvme.sh ################nvme_trtype=loop############ nvme/002 (create many subsystems and test discovery) [passed] runtime ... 19.549s nvme/003 (test if we're sending keep-alives to a discovery controller) [passed] runtime 10.088s ... 10.084s nvme/004 (test nvme and nvmet UUID NS descriptors) [passed] runtime 1.450s ... 1.446s nvme/005 (reset local loopback target) [passed] runtime 1.797s ... 1.801s nvme/006 (create an NVMeOF target with a block device-backed ns) [passed] runtime 0.065s ... 0.061s nvme/007 (create an NVMeOF target with a file-backed ns) [passed] runtime 0.033s ... 0.035s nvme/008 (create an NVMeOF host with a block device-backed ns) [passed] runtime 1.455s ... 1.440s nvme/009 (create an NVMeOF host with a file-backed ns) [passed] runtime 1.425s ... 1.434s nvme/010 (run data verification fio job on NVMeOF block device-backed ns) [passed] runtime 114.095s ... 106.628s nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed] runtime 86.929s ... 77.563s nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed] runtime 96.307s ... 94.556s nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns) [passed] runtime 65.391s ... 70.191s nvme/014 (flush a NVMeOF block device-backed ns) [passed] runtime 7.771s ... 7.219s nvme/015 (unit test for NVMe flush for file backed ns) [passed] runtime 6.432s ... 6.160s nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [passed] runtime 12.562s ... 12.469s nvme/017 (create/delete many file-ns and test discovery) [passed] runtime 12.345s ... 12.581s nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed] runtime 1.436s ... 1.433s nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed] runtime 1.449s ... 1.444s nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed] runtime 1.421s ... 1.412s nvme/021 (test NVMe list command on NVMeOF file-backed ns) [passed] runtime 1.421s ... 1.449s nvme/022 (test NVMe reset command on NVMeOF file-backed ns) [passed] runtime 1.759s ... 1.758s nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed] runtime 1.449s ... 1.458s nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed] runtime 1.418s ... 1.425s nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed] runtime 1.411s ... 1.413s nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed] runtime 1.424s ... 1.428s nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed] runtime 1.430s ... 1.418s nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed] runtime 1.432s ... 1.435s nvme/029 (test userspace IO via nvme-cli read/write interface) [passed] runtime 1.551s ... 1.557s nvme/030 (ensure the discovery generation counter is updated appropriately) [passed] runtime 0.197s ... 0.205s nvme/031 (test deletion of NVMeOF controllers immediately after setup) [passed] runtime 3.895s ... 3.950s nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed] runtime 0.013s ... 0.015s nvme/040 (test nvme fabrics controller reset/disconnect operation during I/O) [passed] runtime 8.069s ... 8.055s nvme/041 (Create authenticated connections) [passed] runtime 0.770s ... 0.751s nvme/042 (Test dhchap key types for authenticated connections) [passed] runtime 4.836s ... 4.890s nvme/043 (Test hash and DH group variations for authenticated connections) [passed] runtime 6.939s ... 7.047s nvme/044 (Test bi-directional authentication) [passed] runtime 1.835s ... 1.834s nvme/045 (Test re-authentication) [passed] runtime 3.812s ... 3.821s nvme/047 (test different queue types for fabric transports) [not run] nvme_trtype=loop is not supported in this test nvme/048 (Test queue count changes on reconnect) [not run] nvme_trtype=loop is not supported in this test blktests (master) # nvme_trtype=tcp ./check nvme nvme/002 (create many subsystems and test discovery) [not run] runtime 19.549s ... nvme_trtype=tcp is not supported in this test nvme/003 (test if we're sending keep-alives to a discovery controller) [passed] runtime 10.084s ... 10.095s nvme/004 (test nvme and nvmet UUID NS descriptors) [passed] runtime 1.446s ... 1.142s nvme/005 (reset local loopback target) [passed] runtime 1.801s ... 1.199s nvme/006 (create an NVMeOF target with a block device-backed ns) [passed] runtime 0.061s ... 0.059s nvme/007 (create an NVMeOF target with a file-backed ns) [passed] runtime 0.035s ... 0.044s nvme/008 (create an NVMeOF host with a block device-backed ns) [passed] runtime 1.440s ... 1.148s nvme/009 (create an NVMeOF host with a file-backed ns) [passed] runtime 1.434s ... 1.139s nvme/010 (run data verification fio job on NVMeOF block device-backed ns) [passed] runtime 106.628s ... 113.605s nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed] runtime 77.563s ... 83.124s nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed] runtime 94.556s ... 86.374s nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns) [passed] runtime 70.191s ... 78.565s nvme/014 (flush a NVMeOF block device-backed ns) [passed] runtime 7.219s ... 6.617s nvme/015 (unit test for NVMe flush for file backed ns) [passed] runtime 6.160s ... 5.939s nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [not run] runtime 12.469s ... nvme_trtype=tcp is not supported in this test nvme/017 (create/delete many file-ns and test discovery) [not run] runtime 12.581s ... nvme_trtype=tcp is not supported in this test nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed] runtime 1.433s ... 1.120s nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed] runtime 1.444s ... 1.148s nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed] runtime 1.412s ... 1.124s nvme/021 (test NVMe list command on NVMeOF file-backed ns) [passed] runtime 1.449s ... 1.127s nvme/022 (test NVMe reset command on NVMeOF file-backed ns) [passed] runtime 1.758s ... 1.164s nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed] runtime 1.458s ... 1.151s nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed] runtime 1.425s ... 1.112s nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed] runtime 1.413s ... 1.121s nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed] runtime 1.428s ... 1.121s nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed] runtime 1.418s ... 1.121s nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed] runtime 1.435s ... 1.130s nvme/029 (test userspace IO via nvme-cli read/write interface) [passed] runtime 1.557s ... 1.261s nvme/030 (ensure the discovery generation counter is updated appropriately) [passed] runtime 0.205s ... 0.115s nvme/031 (test deletion of NVMeOF controllers immediately after setup) [passed] runtime 3.950s ... 0.802s nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed] runtime 0.015s ... 0.015s nvme/040 (test nvme fabrics controller reset/disconnect operation during I/O) [passed] runtime 8.055s ... 7.296s nvme/041 (Create authenticated connections) [passed] runtime 0.751s ... 0.448s nvme/042 (Test dhchap key types for authenticated connections) [passed] runtime 4.890s ... 2.757s nvme/043 (Test hash and DH group variations for authenticated connections) [passed] runtime 7.047s ... 0.705s nvme/044 (Test bi-directional authentication) [passed] runtime 1.834s ... 1.227s nvme/045 (Test re-authentication) [passed] runtime 3.821s ... 3.646s nvme/047 (test different queue types for fabric transports) [passed] runtime ... 1.718s nvme/048 (Test queue count changes on reconnect) [passed] runtime ... 6.242s blktests (master) #