I got it somewhat working with this version here and the blktest fcloop cleanup
fix[1].
There is a BIG if though. I have to disable manually the auto connect udev rules
from the host system. I've played a bit around how to disable them during
runtime. There is a way to inject a runtime global variable into udev rules via
'udevadm control -p BLKTESTS=1'. The udev rule file can then be extended with a
rule like 'ENV{BLKTESTS}="1", GOTO=...' but I was not able to get this working
reliable. It looks like the update is not getting promotted to all udev daemon
threads. Even a 'sleep 5' or 'udevadm settle' didn't help.
The only way I found so far is to do
ln -s /dev/null /etc/udev/rules.d/70-nvmf-autoconnect.rules
which obviously sucks. IIRC, Martin B has the same issue with nvme-stats and
nvme-cli...
Anyway, it doesn't creash anymore and there are not warning from lockdep or
KASAN. The first few tests cases are passing. So the failing ones could actually
be 'real' bugs, who knows :)
# nvme_trtype=fc ./check nvme
nvme/002 (create many subsystems and test discovery) [not run]
nvme_trtype=fc is not supported in this test
nvme/003 (test if we're sending keep-alives to a discovery controller)
nvme/003 (test if we're sending keep-alives to a discovery controller) [passed]
runtime 10.277s ... 10.479srt: No such file or directory
nvme/004 (test nvme and nvmet UUID NS descriptors) [passed]
runtime 1.976s ... 1.723s
nvme/005 (reset local loopback target) [passed]
runtime 2.203s ... 1.829s
nvme/006 (create an NVMeOF target with a block device-backed ns) [passed]
runtime 0.248s ... 0.221s
nvme/007 (create an NVMeOF target with a file-backed ns) [passed]
runtime 0.156s ... 0.127s
nvme/008 (create an NVMeOF host with a block device-backed ns) [passed]
runtime 1.945s ... 1.658s
nvme/009 (create an NVMeOF host with a file-backed ns) [passed]
runtime 1.854s ... 1.711s
nvme/010 (run data verification fio job on NVMeOF block device-backed ns) [passed]
runtime 44.555s ... 41.625s
nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed]
runtime 89.795s ... 71.931s
nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed]
runtime 61.607s ... 49.917s
nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns) [passed]
runtime 109.725s ... 79.189s
nvme/014 (flush a NVMeOF block device-backed ns) [passed]
runtime 7.902s ... 5.977s
nvme/015 (unit test for NVMe flush for file backed ns) [passed]
runtime 7.229s ... 5.734s
nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [not run]
runtime 44.446s ...
nvme_trtype=fc is not supported in this test
nvme/017 (create/delete many file-ns and test discovery) [not run]
runtime 45.306s ...
nvme_trtype=fc is not supported in this test
nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed]
runtime 1.834s ... 1.721s
nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed]
runtime 1.832s ... 1.804s
nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed]
runtime 1.811s ... 1.717s
nvme/021 (test NVMe list command on NVMeOF file-backed ns) [passed]
runtime 1.807s ... 1.703s
nvme/022 (test NVMe reset command on NVMeOF file-backed ns) [passed]
runtime 1.914s ... 1.784s
nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed]
runtime 1.852s ... 1.730s
nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed]
runtime 1.730s ... 1.754s
nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed]
runtime 1.759s ... 1.719s
nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed]
runtime 1.764s ... 1.675s
nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed]
runtime 1.734s ... 1.703s
nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed]
runtime 1.831s ... 1.732s
nvme/029 (test userspace IO via nvme-cli read/write interface) [passed]
runtime 2.388s ... 2.065s
nvme/030 (ensure the discovery generation counter is updated appropriately) [passed]
runtime 0.756s ... 0.631s
nvme/031 (test deletion of NVMeOF controllers immediately after setup) [passed]
runtime 4.566s ... 3.441s
nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed]
runtime 0.054s ... 0.055s
nvme/040 (test nvme fabrics controller reset/disconnect operation during I/O)
nvme/040 (test nvme fabrics controller reset/disconnect operation during I/O) [passed]
runtime 7.948s ... 7.866srk/blktests/results/tmpdir.nvme.040.4lT': Directory not empty
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
modprobe: FATAL: Module loop is in use.
nvme/041 (Create authenticated connections)
runtime 0.964s ...
WARNING: Test did not clean up fc device: nvme0
nvme/041 (Create authenticated connections) [failed]
runtime 0.964s ... 0.311s
--- tests/nvme/041.out 2023-02-20 10:31:10.953935278 +0100
+++ /home/wagi/work/blktests/results/nodev/nvme/041.out.bad 2023-04-18 14:31:05.122062907 +0200
@@ -1,6 +1,8 @@
Running nvme/041
Test unauthenticated connection (should fail)
+failed to lookup subsystem for controller nvme0
NQN:blktests-subsystem-1 disconnected 0 controller(s)
Test authenticated connection
-NQN:blktests-subsystem-1 disconnected 1 controller(s)
+failed to lookup subsystem for controller nvme0
...
(Run 'diff -u tests/nvme/041.out /home/wagi/work/blktests/results/nodev/nvme/041.out.bad' to see the entire diff)
WARNING: Test did not clean up fc device: nvme0
failed to lookup subsystem for controller nvme0
Did not find device nvme0
nvme/042 (Test dhchap key types for authenticated connections) [failed]
runtime 4.495s ... 4.226s
--- tests/nvme/042.out 2022-08-30 10:20:14.174819528 +0200
+++ /home/wagi/work/blktests/results/nodev/nvme/042.out.bad 2023-04-18 14:31:09.654086133 +0200
@@ -1,7 +1,9 @@
Running nvme/042
Testing hmac 0
+failed to lookup subsystem for controller nvme0
NQN:blktests-subsystem-1 disconnected 1 controller(s)
Testing hmac 1
+failed to lookup subsystem for controller nvme0
NQN:blktests-subsystem-1 disconnected 1 controller(s)
...
(Run 'diff -u tests/nvme/042.out /home/wagi/work/blktests/results/nodev/nvme/042.out.bad' to see the entire diff)
nvme/043 (Test hash and DH group variations for authenticated connections) [passed]
runtime 3.592s ... 8.750s
nvme/044 (Test bi-directional authentication)
runtime 1.099s ...
WARNING: Test did not clean up fc device: nvme0
nvme/044 (Test bi-directional authentication) [failed]
runtime 1.099s ... 1.064s
--- tests/nvme/044.out 2023-02-20 10:31:10.953935278 +0100
+++ /home/wagi/work/blktests/results/nodev/nvme/044.out.bad 2023-04-18 14:31:20.042139370 +0200
@@ -2,9 +2,12 @@
Test host authentication
NQN:blktests-subsystem-1 disconnected 1 controller(s)
Test invalid ctrl authentication (should fail)
+failed to lookup subsystem for controller nvme0
NQN:blktests-subsystem-1 disconnected 0 controller(s)
Test valid ctrl authentication
-NQN:blktests-subsystem-1 disconnected 1 controller(s)
...
(Run 'diff -u tests/nvme/044.out /home/wagi/work/blktests/results/nodev/nvme/044.out.bad' to see the entire diff)
WARNING: Test did not clean up fc device: nvme0
failed to lookup subsystem for controller nvme0
Did not find device nvme0
nvme/045 (Test re-authentication) [passed]
runtime 5.582s ... 6.262s
nvme/047 (test different queue types for fabric transports) [not run]
nvme_trtype=fc is not supported in this test
nvme/048 (Test queue count changes on reconnect) [failed]
runtime 15.586s ... 16.545s
--- tests/nvme/048.out 2023-04-06 10:12:58.333064747 +0200
+++ /home/wagi/work/blktests/results/nodev/nvme/048.out.bad 2023-04-18 14:31:43.614260172 +0200
@@ -1,3 +1,7 @@
Running nvme/048
+expected state "connecting" not reached within 5 seconds
+FAIL
+expected state "connecting" not reached within 5 seconds
+FAIL
NQN:blktests-subsystem-1 disconnected 1 controller(s)
Test complete
[1] https://lore.kernel.org/linux-nvme/20230418123252.3725-1-dwagner@xxxxxxx/
changes:
v3:
- do not unlink rport twice
v2:
- added additional fixes
- https://lore.kernel.org/linux-nvme/20230411120718.14477-1-dwagner@xxxxxxx/
v1:
- initial version
- https://lore.kernel.org/linux-nvme/20230411092209.12719-1-dwagner@xxxxxxx/
Daniel Wagner (4):
nvmet-fcloop: Remove remote port from list when unlinking
nvmet-fcloop: Do not wait on completion when unregister fails
nvmet-fc: Do not wait in vain when unloading module
nvmet-fc: Release reference on target port
drivers/nvme/host/fc.c | 20 +++++++++++++-------
drivers/nvme/target/fc.c | 1 +
drivers/nvme/target/fcloop.c | 10 ++++------
3 files changed, 18 insertions(+), 13 deletions(-)
--
2.40.0
