Re: ESX FC host connectivity issues

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Mar 18, 2016 at 4:21 PM, DAVID S <shraderdm@xxxxxxxxx> wrote:
> Hi all,
>
> I have a very similar setup to Dan, and I am also still seeing a large
> number of ABORT_TASK errors.
>
> I am running ESXi 5.5 with VAAI heartbeat turned off, my fibre channel cards
> are all qlogic qle2462 running the most recent firmware, and my target is
> running on kernel 4.5.
>
> Unlike Dan, I have not had any crashes since updating to kernel 4.5 two days
> ago, but I am still seeing these errors. It is very possible that his
> systems are performing more reads and writes, and his storage is much faster
> than mine, so I will keep an eye out and if mine fails I will reply here
> again.
>
> Here are the errors that I'm seeing:
> Mar 18 15:13:14 storage kernel: qla2xxx/21:00:00:24:ff:56:b0:41: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:13:35 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1200416
> Mar 18 15:13:35 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1200416
> Mar 18 15:13:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210008
> Mar 18 15:13:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210008
> Mar 18 15:13:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210140
> Mar 18 15:13:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210140
> Mar 18 15:13:55 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1214408
> Mar 18 15:13:55 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1214408
> Mar 18 15:14:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1221316
> Mar 18 15:14:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1221316
> Mar 18 15:17:58 storage systemd: Starting dnf makecache...
> Mar 18 15:17:58 storage dnf: cachedir: /var/cache/dnf
> Mar 18 15:17:58 storage dnf: Loaded plugins: copr, debuginfo-install,
> protected_packages, generate_completion_cache, Query, download, builddep,
> noroot, config-manager, needs-restarting, langpacks, reposync, playground
> Mar 18 15:17:58 storage dnf: Adding en to language list
> Mar 18 15:17:58 storage dnf: Error reading file :
> /var/lib/dnf/plugins/langpacks/installed_langpacks as it does not exist
> Mar 18 15:17:58 storage dnf: initialized Langpacks plugin
> Mar 18 15:17:58 storage dnf: DNF version: 1.1.7
> Mar 18 15:17:58 storage dnf: Making cache files for all metadata files.
> Mar 18 15:17:58 storage dnf: updates: metadata will expire after 10620
> seconds and will be refreshed now
> Mar 18 15:17:58 storage dnf: fedora: will expire after 576717 seconds.
> Mar 18 15:17:59 storage dnf: reviving: 'updates' can be revived.
> Mar 18 15:17:59 storage dnf: repo: using cache for: fedora
> Mar 18 15:17:59 storage dnf: not found deltainfo for: Fedora 23 - x86_64
> Mar 18 15:17:59 storage dnf: not found updateinfo for: Fedora 23 - x86_64
> Mar 18 15:17:59 storage dnf: Metadata cache created.
> Mar 18 15:18:00 storage systemd: Started dnf makecache.
> Mar 18 15:18:00 storage audit: SERVICE_START pid=1 uid=0 auid=4294967295
> ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-makecache
> comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=?
> res=success'
> Mar 18 15:18:00 storage audit: SERVICE_STOP pid=1 uid=0 auid=4294967295
> ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-makecache
> comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=?
> res=success'
> Mar 18 15:20:30 storage kernel: qla2xxx/21:01:00:e0:8b:bd:5f:4f: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:20:30 storage kernel: qla2xxx/21:00:00:e0:8b:9d:5f:4f: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:20:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ad: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:20:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ac: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:23:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1168076
> Mar 18 15:23:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1168076
> Mar 18 15:23:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1168120
> Mar 18 15:23:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1168120
> Mar 18 15:24:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1184620
> Mar 18 15:24:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1184620
> Mar 18 15:24:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1184664
> Mar 18 15:24:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1184664
> Mar 18 15:24:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1201296
> Mar 18 15:24:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1201296
> Mar 18 15:24:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1156152
> Mar 18 15:24:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1156152
> Mar 18 15:25:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1176524
> Mar 18 15:25:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1176524
> Mar 18 15:25:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1195664
> Mar 18 15:25:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1195664
> Mar 18 15:25:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1209436
> Mar 18 15:25:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1209436
> Mar 18 15:28:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144844
> Mar 18 15:28:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144844
> Mar 18 15:28:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144888
> Mar 18 15:28:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144888
> Mar 18 15:29:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1162620
> Mar 18 15:29:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1162620
> Mar 18 15:29:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1162664
> Mar 18 15:29:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1162664
> Mar 18 15:29:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1191792
> Mar 18 15:29:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1191792
> Mar 18 15:33:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1153204
> Mar 18 15:33:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1153204
> Mar 18 15:33:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1153380
> Mar 18 15:33:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1153380
> Mar 18 15:33:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1153468
> Mar 18 15:33:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1153468
> Mar 18 15:33:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1153996
> Mar 18 15:33:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1153996
> Mar 18 15:33:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1154084
> Mar 18 15:33:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1154084
> Mar 18 15:33:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1154040
> Mar 18 15:33:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1154040
> Mar 18 15:33:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1155096
> Mar 18 15:33:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1155096
> Mar 18 15:33:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1155404
> Mar 18 15:33:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1155404
> Mar 18 15:33:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1155448
> Mar 18 15:33:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1155448
> Mar 18 15:33:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1155932
> Mar 18 15:33:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1156020
> Mar 18 15:33:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1155932
> Mar 18 15:33:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1156020
> Mar 18 15:33:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1155976
> Mar 18 15:33:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1155976
> Mar 18 15:33:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1156460
> Mar 18 15:33:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1156460
> Mar 18 15:33:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1156372
> Mar 18 15:33:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1156372
> Mar 18 15:33:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1156416
> Mar 18 15:33:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1156416
> Mar 18 15:33:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1157296
> Mar 18 15:33:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1157296
> Mar 18 15:33:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1157340
> Mar 18 15:33:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1157340
> Mar 18 15:33:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1157252
> Mar 18 15:33:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1157252
> Mar 18 15:33:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1158132
> Mar 18 15:33:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1158132
> Mar 18 15:33:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1180748
> Mar 18 15:33:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1180748
> Mar 18 15:34:01 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1204156
> Mar 18 15:34:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1204156
> Mar 18 15:35:07 storage kernel: qla2xxx/50:01:43:80:06:37:c4:14: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:35:07 storage kernel: qla2xxx/50:01:43:80:06:37:c4:16: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:38:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1133668
> Mar 18 15:38:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1133668
> Mar 18 15:38:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1134064
> Mar 18 15:38:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1134064
> Mar 18 15:38:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1134020
> Mar 18 15:38:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1134020
> Mar 18 15:38:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1134328
> Mar 18 15:38:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1134328
> Mar 18 15:38:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1135076
> Mar 18 15:38:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1135076
> Mar 18 15:38:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1135032
> Mar 18 15:38:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1135032
> Mar 18 15:38:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1135120
> Mar 18 15:38:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1135120
> Mar 18 15:38:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1135604
> Mar 18 15:38:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1135604
> Mar 18 15:38:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1135868
> Mar 18 15:38:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1135868
> Mar 18 15:38:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1135824
> Mar 18 15:38:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1135824
> Mar 18 15:38:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1135780
> Mar 18 15:38:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1135780
> Mar 18 15:38:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1136440
> Mar 18 15:38:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1136440
> Mar 18 15:38:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1136484
> Mar 18 15:38:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1136484
> Mar 18 15:38:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1136396
> Mar 18 15:38:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1136396
> Mar 18 15:38:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1148276
> Mar 18 15:38:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1148276
> Mar 18 15:39:01 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1171068
> Mar 18 15:39:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1171068
> Mar 18 15:39:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1193948
> Mar 18 15:39:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1193948
> Mar 18 15:39:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1194520
> Mar 18 15:39:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1194520
> Mar 18 15:39:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1195048
> Mar 18 15:39:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1195048
> Mar 18 15:39:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1196192
> Mar 18 15:39:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1196192
> Mar 18 15:40:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1197600
> Mar 18 15:40:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1197600
> Mar 18 15:40:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1198568
> Mar 18 15:40:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1198568
> Mar 18 15:40:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1199624
> Mar 18 15:40:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1199624
> Mar 18 15:41:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1200768
> Mar 18 15:41:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1200768
> Mar 18 15:41:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1202000
> Mar 18 15:41:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1202000
> Mar 18 15:41:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1203364
> Mar 18 15:41:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1203364
> Mar 18 15:43:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1208116
> Mar 18 15:43:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1208116
> Mar 18 15:43:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1208292
> Mar 18 15:43:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1208292
> Mar 18 15:43:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1208160
> Mar 18 15:43:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1208160
> Mar 18 15:43:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1208644
> Mar 18 15:43:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1208644
> Mar 18 15:43:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1208864
> Mar 18 15:43:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1208864
> Mar 18 15:43:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1208688
> Mar 18 15:43:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1208688
> Mar 18 15:43:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1209876
> Mar 18 15:43:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1209876
> Mar 18 15:43:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1209964
> Mar 18 15:43:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1209964
> Mar 18 15:43:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210008
> Mar 18 15:43:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210008
> Mar 18 15:43:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210668
> Mar 18 15:43:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210756
> Mar 18 15:43:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210756
> Mar 18 15:43:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210668
> Mar 18 15:43:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210712
> Mar 18 15:43:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210712
> Mar 18 15:43:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1211196
> Mar 18 15:43:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1211108
> Mar 18 15:43:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1211196
> Mar 18 15:43:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1211108
> Mar 18 15:43:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1211152
> Mar 18 15:43:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1211152
> Mar 18 15:43:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1212032
> Mar 18 15:43:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1212032
> Mar 18 15:43:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1211988
> Mar 18 15:43:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1211988
> Mar 18 15:43:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1212076
> Mar 18 15:43:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1212076
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1212912
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1212912
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1147616
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1147616
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1133756
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1133756
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1133888
> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1133888
> Mar 18 15:44:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1151136
> Mar 18 15:44:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1151136
> Mar 18 15:44:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1151180
> Mar 18 15:44:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1151180
> Mar 18 15:44:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1159760
> Mar 18 15:44:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1159760
> Mar 18 15:44:16 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1168780
> Mar 18 15:44:16 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1168780
> Mar 18 15:44:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1189064
> Mar 18 15:44:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1189064
> Mar 18 15:44:35 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1208908
> Mar 18 15:44:35 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1208908
> Mar 18 15:44:36 storage kernel: qla2xxx/21:00:00:24:ff:56:b0:41: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:48:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1175864
> Mar 18 15:48:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1175864
> Mar 18 15:48:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1175952
> Mar 18 15:48:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1175952
> Mar 18 15:49:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1186468
> Mar 18 15:49:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1186468
> Mar 18 15:49:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1186424
> Mar 18 15:49:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1186424
> Mar 18 15:49:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1215816
> Mar 18 15:49:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1215816
> Mar 18 15:50:30 storage kernel: qla2xxx/21:01:00:e0:8b:bd:5f:4f: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:50:30 storage kernel: qla2xxx/21:01:00:e0:8b:bd:5f:4f: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:50:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ad: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:50:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ad: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 15:53:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1137672
> Mar 18 15:53:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1137672
> Mar 18 15:53:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1137584
> Mar 18 15:53:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1137584
> Mar 18 15:53:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1137628
> Mar 18 15:53:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1137628
> Mar 18 15:53:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1138200
> Mar 18 15:53:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1138200
> Mar 18 15:53:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1138464
> Mar 18 15:53:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1138464
> Mar 18 15:53:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1138376
> Mar 18 15:53:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1138376
> Mar 18 15:53:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1139168
> Mar 18 15:53:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1139168
> Mar 18 15:53:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1139124
> Mar 18 15:53:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1139124
> Mar 18 15:53:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1139212
> Mar 18 15:53:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1139212
> Mar 18 15:53:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1139872
> Mar 18 15:53:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1139872
> Mar 18 15:53:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1139916
> Mar 18 15:53:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1139916
> Mar 18 15:53:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1139960
> Mar 18 15:53:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1139960
> Mar 18 15:53:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1140268
> Mar 18 15:53:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1140268
> Mar 18 15:53:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1140356
> Mar 18 15:53:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1140356
> Mar 18 15:53:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1140312
> Mar 18 15:53:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1140312
> Mar 18 15:53:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1141104
> Mar 18 15:53:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1141104
> Mar 18 15:53:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1141060
> Mar 18 15:53:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1141016
> Mar 18 15:53:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1141060
> Mar 18 15:53:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1141016
> Mar 18 15:53:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1141808
> Mar 18 15:53:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1141808
> Mar 18 15:53:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1164468
> Mar 18 15:53:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1164468
> Mar 18 15:54:01 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1187304
> Mar 18 15:54:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1187304
> Mar 18 15:54:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210096
> Mar 18 15:54:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210096
> Mar 18 15:54:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1210448
> Mar 18 15:54:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1210448
> Mar 18 15:54:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1211152
> Mar 18 15:54:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1211152
> Mar 18 15:54:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1211636
> Mar 18 15:54:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1211636
> Mar 18 15:54:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1211988
> Mar 18 15:54:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1211988
> Mar 18 15:54:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1213220
> Mar 18 15:54:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1213220
> Mar 18 15:54:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1213528
> Mar 18 15:54:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1213528
> Mar 18 15:55:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1214628
> Mar 18 15:55:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1214628
> Mar 18 15:55:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1215024
> Mar 18 15:55:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1215024
> Mar 18 15:55:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1216036
> Mar 18 15:55:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1216036
> Mar 18 15:55:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1216344
> Mar 18 15:55:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1216344
> Mar 18 16:03:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1142380
> Mar 18 16:03:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1142380
> Mar 18 16:03:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1142292
> Mar 18 16:03:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1142292
> Mar 18 16:03:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1142644
> Mar 18 16:03:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1142644
> Mar 18 16:03:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1142864
> Mar 18 16:03:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1142864
> Mar 18 16:03:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1142952
> Mar 18 16:03:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1142952
> Mar 18 16:03:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1143216
> Mar 18 16:03:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1143216
> Mar 18 16:03:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1143832
> Mar 18 16:03:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1143832
> Mar 18 16:03:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144140
> Mar 18 16:03:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144140
> Mar 18 16:03:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144184
> Mar 18 16:03:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144184
> Mar 18 16:03:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144580
> Mar 18 16:03:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144580
> Mar 18 16:03:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144536
> Mar 18 16:03:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144536
> Mar 18 16:03:29 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144492
> Mar 18 16:03:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144492
> Mar 18 16:03:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144932
> Mar 18 16:03:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144932
> Mar 18 16:03:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144844
> Mar 18 16:03:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144844
> Mar 18 16:03:32 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1144888
> Mar 18 16:03:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1144888
> Mar 18 16:03:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1145548
> Mar 18 16:03:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1145548
> Mar 18 16:03:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1145592
> Mar 18 16:03:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1145592
> Mar 18 16:03:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1145504
> Mar 18 16:03:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1145504
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1146780
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1146780
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1215552
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1215552
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1215596
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1215596
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1192760
> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1192760
> Mar 18 16:03:52 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1168956
> Mar 18 16:03:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1168956
> Mar 18 16:04:01 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1191924
> Mar 18 16:04:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1191924
> Mar 18 16:04:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1214276
> Mar 18 16:04:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1214276
> Mar 18 16:04:09 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1204948
> Mar 18 16:04:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1204948
> Mar 18 16:04:12 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1214584
> Mar 18 16:04:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1214584
> Mar 18 16:04:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1215200
> Mar 18 16:04:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1215200
> Mar 18 16:04:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1216300
> Mar 18 16:04:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1216300
> Mar 18 16:05:01 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1217356
> Mar 18 16:05:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1217356
> Mar 18 16:05:07 storage kernel: qla2xxx/50:01:43:80:06:37:c4:14: Unsupported
> SCSI Opcode 0x85, sending CHECK_CONDITION.
> Mar 18 16:05:21 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1218456
> Mar 18 16:05:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1218456
> Mar 18 16:05:28 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1218852
> Mar 18 16:05:28 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1218852
> Mar 18 16:05:41 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1219424
> Mar 18 16:05:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1219424
> Mar 18 16:05:47 storage kernel: ABORT_TASK: Found referenced qla2xxx
> task_tag: 1220084
> Mar 18 16:05:47 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST
> for ref_tag: 1220084
>
>
> I have grepped out the NON_EXISTENT_LUN errors, as those are expected and
> would make it more difficult to see the relevant logs.
>
> Please let me know if I can do anything else to help out.
>
> Thanks!
> David
>
> On Thu, Mar 17, 2016 at 1:23 PM, Dan Lane <dracodan@xxxxxxxxx> wrote:
>>
>> On Wed, Mar 16, 2016 at 5:08 PM, Dan Lane <dracodan@xxxxxxxxx> wrote:
>> > On Wed, Mar 16, 2016 at 10:47 AM, Dan Lane <dracodan@xxxxxxxxx> wrote:
>> >> On Tue, Mar 15, 2016 at 3:49 PM, Dan Lane <dracodan@xxxxxxxxx> wrote:
>> >>> On Tue, Mar 15, 2016 at 3:45 PM, Dan Lane <dracodan@xxxxxxxxx> wrote:
>> >>>> I went to pull the latest source and I noticed mainline Kernel 4.5
>> >>>> was
>> >>>> released yesterday.  Did all the recent patches that apply to fiber
>> >>>> channel make it into this release or do I still need to patch?
>> >>>>
>> >>>> Thanks
>> >>>> Dan
>> >>>>
>> >>>> On Fri, Mar 11, 2016 at 11:07 PM, Nicholas A. Bellinger
>> >>>> <nab@xxxxxxxxxxxxxxx> wrote:
>> >>>>> On Fri, 2016-03-11 at 18:15 -0500, Dan Lane wrote:
>> >>>>>> I'm back in town now and ready to try this again.  Should I still
>> >>>>>> try
>> >>>>>> this patch?
>> >>>>>
>> >>>>> Yes, you still need to apply the patch to drop the extra bogus
>> >>>>> target_put_sess_cmd() call, when !__target_check_io_state() for
>> >>>>> ABORT_TASK occurs:
>> >>>>>
>> >>>>>
>> >>>>> https://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?id=7f54ab5ff52fb0b91569bc69c4a6bc5cac1b768d
>> >>>>>
>> >>>>>> I noticed you had submitted a patch a few days ago, so
>> >>>>>> can I just pull all the latest updates from your git repo?
>> >>>>>>
>> >>>>>
>> >>>>> The PULL request just went out to Linus, and will be included for
>> >>>>> v4.5 release.
>> >>>>>
>> >>>>> It's also CC'ed for stable, and will make it's way down to v3.14.y
>> >>>>> stable over the next weeks.
>> >>>>>
>> >>>
>> >>> Whoops, sorry about the top post...  I know you said the request was
>> >>> sent to Linus, I just wanted to confirm that it made it since the time
>> >>> frame between that last email and when 4.5 was released was so short.
>> >>>
>> >>> Thanks again
>> >>
>> >> Latest update:
>> >> With the 4.5 (final) kernel my storage was more stable than ever, but
>> >> again went inaccessible after about 15 hours.  This is despite very
>> >> heavy usage last night, the likes of which caused failures in the past
>> >> (but I was amazed with the performance, I was able to get 650MB/s
>> >> writes and 750MB/s reads!!!).  The aborts seem to be coming in as
>> >> steady as they have in the past, which leads me to believe the patch
>> >> for the extra "bogus target_put_sess_cmd() call" didn't make it in
>> >> time for the 4.5 release.  If it did, this means there are more
>> >> problems.
>> >>
>> >> Here is a snippet from my messages log before ESXi gave up:
>> >> Mar 16 07:21:57 dracofiler kernel: ABORT_TASK: Sending
>> >> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1169660
>> >> Mar 16 07:21:57 dracofiler kernel: ABORT_TASK: Found referenced
>> >> qla2xxx task_tag: 1169616
>> >> Mar 16 07:21:57 dracofiler kernel: ABORT_TASK: Sending
>> >> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1169616
>> >> Mar 16 07:23:20 dracofiler kernel: ABORT_TASK: Found referenced
>> >> qla2xxx task_tag: 1147132
>> >> Mar 16 07:23:20 dracofiler kernel: ABORT_TASK: Sending
>> >> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1147132
>> >> Mar 16 07:23:20 dracofiler kernel: ABORT_TASK: Found referenced
>> >> qla2xxx task_tag: 1147176
>> >> Mar 16 07:23:24 dracofiler kernel: ABORT_TASK: Sending
>> >> TMR_FUNCTION_COMPLETE for ref_tag: 1147176
>> >> Mar 16 07:23:24 dracofiler kernel: ABORT_TASK: Found referenced
>> >> qla2xxx task_tag: 1186556
>> >>
>> >> Also, I configured my hosts to send their logs to a syslog server, I
>> >> have an appointment to go to but I'll pull those and send them to you
>> >> this afternoon.
>> >>
>> >> Thanks
>> >> Dan
>> >
>> > I discovered the ATS heartbeat issue was still causing issues.  I have
>> > created a host profile and applied it to all of my hosts to ensure it
>> > doesn't come up again.  For now there's no reason to dig further with
>> > this, I will report back whether or not I'm still having the issue in
>> > the next few days (or sooner if it still fails).
>> >
>> > Thanks,
>> > Dan
>>
>>
>> Okay, back on track, I'm still seeing these aborts and eventually
>> losing access to the storage despite running the final 4.5 kernel and
>> VMFS3.UseATSForHBonVMFS5=0 set on all hosts.  According to the log and
>> what you have explained in the past, I think it still looks like I'm
>> using ATS heartbeat, but I may be wrong.  Note, it takes a lot longer
>> to fail than it used to, but I can still trigger the failure by
>> running ATTO repeatedly from a VM.  Also, for comparing between the
>> logs, my timezone is GMT/Zulu -4 (target server is local time,
>> vmkernel.log is zulu).
>>
>> Here is my target log from the time period when it finally failed
>> (atto was running from a VM at this time):
>> Mar 16 23:30:49 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1165392
>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1165744
>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1165744
>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1165832
>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1165832
>> Mar 16 23:31:06 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1180616
>> Mar 16 23:31:06 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1180616
>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1182244
>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182244
>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1182200
>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182200
>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1182552
>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182552
>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1182508
>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182508
>> Mar 16 23:34:18 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1152236
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_FUNCTION_COMPLETE for ref_tag: 1152236
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1161124
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1161124
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1199888
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1199888
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1156680
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1166976
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1168164
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1168164
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1156680
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1156680
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1172784
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1134064
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1134064
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1174720
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1174720
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1134152
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1134152
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1215288
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1185368
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1185632
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1223648
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1192012
>> Mar 16 23:34:28 dracofiler kernel: Detected MISCOMPARE for addr:
>> ffff88062f84c000 buf: ffff88062d360c00
>> Mar 16 23:34:28 dracofiler kernel: Target/iblock: Send MISCOMPARE
>> check condition and sense
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_FUNCTION_COMPLETE for ref_tag: 1192012
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1201912
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1143128
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1176524
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1172476
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1135824
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1169044
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1173136
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1174060
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1198084
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1174192
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1187084
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1146560
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1146604
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1196148
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1152280
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1144844
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1145768
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1146120
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1191748
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1159496
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1159540
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1170848
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1219336
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1219424
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1179692
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1208600
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186028
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186160
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186204
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186248
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1193508
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1200548
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1151796
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1155712
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1185852
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1200284
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1219028
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1139300
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1150168
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1150212
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1136000
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending
>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1136000
>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced
>> qla2xxx task_tag: 1218896
>>
>>
>> Dan
>
>

So.... is someone looking into this?  Is there anything else we can
contribute to help?

Thanks
Dan
--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux SCSI]     [Kernel Newbies]     [Linux SCSI Target Infrastructure]     [Share Photos]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Linux IIO]     [Device Mapper]

  Powered by Linux