On Mon, Mar 21, 2016 at 11:48 PM, Dan Lane <dracodan@xxxxxxxxx> wrote: > On Fri, Mar 18, 2016 at 4:21 PM, DAVID S <shraderdm@xxxxxxxxx> wrote: >> Hi all, >> >> I have a very similar setup to Dan, and I am also still seeing a large >> number of ABORT_TASK errors. >> >> I am running ESXi 5.5 with VAAI heartbeat turned off, my fibre channel cards >> are all qlogic qle2462 running the most recent firmware, and my target is >> running on kernel 4.5. >> >> Unlike Dan, I have not had any crashes since updating to kernel 4.5 two days >> ago, but I am still seeing these errors. It is very possible that his >> systems are performing more reads and writes, and his storage is much faster >> than mine, so I will keep an eye out and if mine fails I will reply here >> again. >> >> Here are the errors that I'm seeing: >> Mar 18 15:13:14 storage kernel: qla2xxx/21:00:00:24:ff:56:b0:41: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:13:35 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1200416 >> Mar 18 15:13:35 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1200416 >> Mar 18 15:13:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210008 >> Mar 18 15:13:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210008 >> Mar 18 15:13:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210140 >> Mar 18 15:13:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210140 >> Mar 18 15:13:55 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1214408 >> Mar 18 15:13:55 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1214408 >> Mar 18 15:14:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1221316 >> Mar 18 15:14:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1221316 >> Mar 18 15:17:58 storage systemd: Starting dnf makecache... >> Mar 18 15:17:58 storage dnf: cachedir: /var/cache/dnf >> Mar 18 15:17:58 storage dnf: Loaded plugins: copr, debuginfo-install, >> protected_packages, generate_completion_cache, Query, download, builddep, >> noroot, config-manager, needs-restarting, langpacks, reposync, playground >> Mar 18 15:17:58 storage dnf: Adding en to language list >> Mar 18 15:17:58 storage dnf: Error reading file : >> /var/lib/dnf/plugins/langpacks/installed_langpacks as it does not exist >> Mar 18 15:17:58 storage dnf: initialized Langpacks plugin >> Mar 18 15:17:58 storage dnf: DNF version: 1.1.7 >> Mar 18 15:17:58 storage dnf: Making cache files for all metadata files. >> Mar 18 15:17:58 storage dnf: updates: metadata will expire after 10620 >> seconds and will be refreshed now >> Mar 18 15:17:58 storage dnf: fedora: will expire after 576717 seconds. >> Mar 18 15:17:59 storage dnf: reviving: 'updates' can be revived. >> Mar 18 15:17:59 storage dnf: repo: using cache for: fedora >> Mar 18 15:17:59 storage dnf: not found deltainfo for: Fedora 23 - x86_64 >> Mar 18 15:17:59 storage dnf: not found updateinfo for: Fedora 23 - x86_64 >> Mar 18 15:17:59 storage dnf: Metadata cache created. >> Mar 18 15:18:00 storage systemd: Started dnf makecache. >> Mar 18 15:18:00 storage audit: SERVICE_START pid=1 uid=0 auid=4294967295 >> ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-makecache >> comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? >> res=success' >> Mar 18 15:18:00 storage audit: SERVICE_STOP pid=1 uid=0 auid=4294967295 >> ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-makecache >> comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? >> res=success' >> Mar 18 15:20:30 storage kernel: qla2xxx/21:01:00:e0:8b:bd:5f:4f: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:20:30 storage kernel: qla2xxx/21:00:00:e0:8b:9d:5f:4f: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:20:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ad: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:20:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ac: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:23:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1168076 >> Mar 18 15:23:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1168076 >> Mar 18 15:23:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1168120 >> Mar 18 15:23:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1168120 >> Mar 18 15:24:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1184620 >> Mar 18 15:24:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1184620 >> Mar 18 15:24:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1184664 >> Mar 18 15:24:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1184664 >> Mar 18 15:24:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1201296 >> Mar 18 15:24:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1201296 >> Mar 18 15:24:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1156152 >> Mar 18 15:24:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1156152 >> Mar 18 15:25:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1176524 >> Mar 18 15:25:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1176524 >> Mar 18 15:25:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1195664 >> Mar 18 15:25:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1195664 >> Mar 18 15:25:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1209436 >> Mar 18 15:25:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1209436 >> Mar 18 15:28:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144844 >> Mar 18 15:28:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144844 >> Mar 18 15:28:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144888 >> Mar 18 15:28:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144888 >> Mar 18 15:29:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1162620 >> Mar 18 15:29:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1162620 >> Mar 18 15:29:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1162664 >> Mar 18 15:29:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1162664 >> Mar 18 15:29:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1191792 >> Mar 18 15:29:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1191792 >> Mar 18 15:33:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1153204 >> Mar 18 15:33:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1153204 >> Mar 18 15:33:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1153380 >> Mar 18 15:33:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1153380 >> Mar 18 15:33:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1153468 >> Mar 18 15:33:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1153468 >> Mar 18 15:33:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1153996 >> Mar 18 15:33:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1153996 >> Mar 18 15:33:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1154084 >> Mar 18 15:33:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1154084 >> Mar 18 15:33:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1154040 >> Mar 18 15:33:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1154040 >> Mar 18 15:33:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1155096 >> Mar 18 15:33:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1155096 >> Mar 18 15:33:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1155404 >> Mar 18 15:33:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1155404 >> Mar 18 15:33:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1155448 >> Mar 18 15:33:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1155448 >> Mar 18 15:33:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1155932 >> Mar 18 15:33:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1156020 >> Mar 18 15:33:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1155932 >> Mar 18 15:33:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1156020 >> Mar 18 15:33:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1155976 >> Mar 18 15:33:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1155976 >> Mar 18 15:33:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1156460 >> Mar 18 15:33:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1156460 >> Mar 18 15:33:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1156372 >> Mar 18 15:33:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1156372 >> Mar 18 15:33:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1156416 >> Mar 18 15:33:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1156416 >> Mar 18 15:33:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1157296 >> Mar 18 15:33:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1157296 >> Mar 18 15:33:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1157340 >> Mar 18 15:33:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1157340 >> Mar 18 15:33:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1157252 >> Mar 18 15:33:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1157252 >> Mar 18 15:33:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1158132 >> Mar 18 15:33:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1158132 >> Mar 18 15:33:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1180748 >> Mar 18 15:33:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1180748 >> Mar 18 15:34:01 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1204156 >> Mar 18 15:34:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1204156 >> Mar 18 15:35:07 storage kernel: qla2xxx/50:01:43:80:06:37:c4:14: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:35:07 storage kernel: qla2xxx/50:01:43:80:06:37:c4:16: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:38:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1133668 >> Mar 18 15:38:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1133668 >> Mar 18 15:38:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1134064 >> Mar 18 15:38:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1134064 >> Mar 18 15:38:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1134020 >> Mar 18 15:38:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1134020 >> Mar 18 15:38:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1134328 >> Mar 18 15:38:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1134328 >> Mar 18 15:38:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1135076 >> Mar 18 15:38:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1135076 >> Mar 18 15:38:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1135032 >> Mar 18 15:38:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1135032 >> Mar 18 15:38:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1135120 >> Mar 18 15:38:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1135120 >> Mar 18 15:38:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1135604 >> Mar 18 15:38:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1135604 >> Mar 18 15:38:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1135868 >> Mar 18 15:38:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1135868 >> Mar 18 15:38:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1135824 >> Mar 18 15:38:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1135824 >> Mar 18 15:38:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1135780 >> Mar 18 15:38:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1135780 >> Mar 18 15:38:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1136440 >> Mar 18 15:38:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1136440 >> Mar 18 15:38:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1136484 >> Mar 18 15:38:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1136484 >> Mar 18 15:38:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1136396 >> Mar 18 15:38:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1136396 >> Mar 18 15:38:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1148276 >> Mar 18 15:38:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1148276 >> Mar 18 15:39:01 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1171068 >> Mar 18 15:39:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1171068 >> Mar 18 15:39:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1193948 >> Mar 18 15:39:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1193948 >> Mar 18 15:39:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1194520 >> Mar 18 15:39:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1194520 >> Mar 18 15:39:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1195048 >> Mar 18 15:39:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1195048 >> Mar 18 15:39:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1196192 >> Mar 18 15:39:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1196192 >> Mar 18 15:40:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1197600 >> Mar 18 15:40:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1197600 >> Mar 18 15:40:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1198568 >> Mar 18 15:40:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1198568 >> Mar 18 15:40:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1199624 >> Mar 18 15:40:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1199624 >> Mar 18 15:41:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1200768 >> Mar 18 15:41:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1200768 >> Mar 18 15:41:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1202000 >> Mar 18 15:41:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1202000 >> Mar 18 15:41:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1203364 >> Mar 18 15:41:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1203364 >> Mar 18 15:43:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1208116 >> Mar 18 15:43:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1208116 >> Mar 18 15:43:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1208292 >> Mar 18 15:43:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1208292 >> Mar 18 15:43:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1208160 >> Mar 18 15:43:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1208160 >> Mar 18 15:43:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1208644 >> Mar 18 15:43:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1208644 >> Mar 18 15:43:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1208864 >> Mar 18 15:43:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1208864 >> Mar 18 15:43:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1208688 >> Mar 18 15:43:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1208688 >> Mar 18 15:43:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1209876 >> Mar 18 15:43:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1209876 >> Mar 18 15:43:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1209964 >> Mar 18 15:43:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1209964 >> Mar 18 15:43:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210008 >> Mar 18 15:43:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210008 >> Mar 18 15:43:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210668 >> Mar 18 15:43:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210756 >> Mar 18 15:43:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210756 >> Mar 18 15:43:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210668 >> Mar 18 15:43:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210712 >> Mar 18 15:43:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210712 >> Mar 18 15:43:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1211196 >> Mar 18 15:43:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1211108 >> Mar 18 15:43:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1211196 >> Mar 18 15:43:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1211108 >> Mar 18 15:43:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1211152 >> Mar 18 15:43:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1211152 >> Mar 18 15:43:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1212032 >> Mar 18 15:43:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1212032 >> Mar 18 15:43:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1211988 >> Mar 18 15:43:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1211988 >> Mar 18 15:43:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1212076 >> Mar 18 15:43:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1212076 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1212912 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1212912 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1147616 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1147616 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1133756 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1133756 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1133888 >> Mar 18 15:43:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1133888 >> Mar 18 15:44:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1151136 >> Mar 18 15:44:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1151136 >> Mar 18 15:44:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1151180 >> Mar 18 15:44:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1151180 >> Mar 18 15:44:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1159760 >> Mar 18 15:44:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1159760 >> Mar 18 15:44:16 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1168780 >> Mar 18 15:44:16 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1168780 >> Mar 18 15:44:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1189064 >> Mar 18 15:44:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1189064 >> Mar 18 15:44:35 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1208908 >> Mar 18 15:44:35 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1208908 >> Mar 18 15:44:36 storage kernel: qla2xxx/21:00:00:24:ff:56:b0:41: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:48:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1175864 >> Mar 18 15:48:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1175864 >> Mar 18 15:48:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1175952 >> Mar 18 15:48:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1175952 >> Mar 18 15:49:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1186468 >> Mar 18 15:49:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1186468 >> Mar 18 15:49:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1186424 >> Mar 18 15:49:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1186424 >> Mar 18 15:49:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1215816 >> Mar 18 15:49:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1215816 >> Mar 18 15:50:30 storage kernel: qla2xxx/21:01:00:e0:8b:bd:5f:4f: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:50:30 storage kernel: qla2xxx/21:01:00:e0:8b:bd:5f:4f: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:50:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ad: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:50:33 storage kernel: qla2xxx/21:00:00:24:ff:4c:37:ad: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 15:53:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1137672 >> Mar 18 15:53:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1137672 >> Mar 18 15:53:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1137584 >> Mar 18 15:53:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1137584 >> Mar 18 15:53:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1137628 >> Mar 18 15:53:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1137628 >> Mar 18 15:53:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1138200 >> Mar 18 15:53:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1138200 >> Mar 18 15:53:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1138464 >> Mar 18 15:53:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1138464 >> Mar 18 15:53:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1138376 >> Mar 18 15:53:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1138376 >> Mar 18 15:53:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1139168 >> Mar 18 15:53:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1139168 >> Mar 18 15:53:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1139124 >> Mar 18 15:53:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1139124 >> Mar 18 15:53:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1139212 >> Mar 18 15:53:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1139212 >> Mar 18 15:53:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1139872 >> Mar 18 15:53:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1139872 >> Mar 18 15:53:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1139916 >> Mar 18 15:53:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1139916 >> Mar 18 15:53:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1139960 >> Mar 18 15:53:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1139960 >> Mar 18 15:53:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1140268 >> Mar 18 15:53:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1140268 >> Mar 18 15:53:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1140356 >> Mar 18 15:53:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1140356 >> Mar 18 15:53:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1140312 >> Mar 18 15:53:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1140312 >> Mar 18 15:53:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1141104 >> Mar 18 15:53:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1141104 >> Mar 18 15:53:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1141060 >> Mar 18 15:53:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1141016 >> Mar 18 15:53:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1141060 >> Mar 18 15:53:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1141016 >> Mar 18 15:53:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1141808 >> Mar 18 15:53:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1141808 >> Mar 18 15:53:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1164468 >> Mar 18 15:53:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1164468 >> Mar 18 15:54:01 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1187304 >> Mar 18 15:54:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1187304 >> Mar 18 15:54:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210096 >> Mar 18 15:54:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210096 >> Mar 18 15:54:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1210448 >> Mar 18 15:54:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1210448 >> Mar 18 15:54:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1211152 >> Mar 18 15:54:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1211152 >> Mar 18 15:54:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1211636 >> Mar 18 15:54:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1211636 >> Mar 18 15:54:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1211988 >> Mar 18 15:54:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1211988 >> Mar 18 15:54:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1213220 >> Mar 18 15:54:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1213220 >> Mar 18 15:54:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1213528 >> Mar 18 15:54:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1213528 >> Mar 18 15:55:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1214628 >> Mar 18 15:55:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1214628 >> Mar 18 15:55:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1215024 >> Mar 18 15:55:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1215024 >> Mar 18 15:55:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1216036 >> Mar 18 15:55:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1216036 >> Mar 18 15:55:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1216344 >> Mar 18 15:55:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1216344 >> Mar 18 16:03:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1142380 >> Mar 18 16:03:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1142380 >> Mar 18 16:03:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1142292 >> Mar 18 16:03:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1142292 >> Mar 18 16:03:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1142644 >> Mar 18 16:03:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1142644 >> Mar 18 16:03:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1142864 >> Mar 18 16:03:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1142864 >> Mar 18 16:03:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1142952 >> Mar 18 16:03:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1142952 >> Mar 18 16:03:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1143216 >> Mar 18 16:03:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1143216 >> Mar 18 16:03:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1143832 >> Mar 18 16:03:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1143832 >> Mar 18 16:03:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144140 >> Mar 18 16:03:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144140 >> Mar 18 16:03:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144184 >> Mar 18 16:03:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144184 >> Mar 18 16:03:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144580 >> Mar 18 16:03:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144580 >> Mar 18 16:03:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144536 >> Mar 18 16:03:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144536 >> Mar 18 16:03:29 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144492 >> Mar 18 16:03:29 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144492 >> Mar 18 16:03:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144932 >> Mar 18 16:03:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144932 >> Mar 18 16:03:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144844 >> Mar 18 16:03:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144844 >> Mar 18 16:03:32 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1144888 >> Mar 18 16:03:32 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1144888 >> Mar 18 16:03:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1145548 >> Mar 18 16:03:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1145548 >> Mar 18 16:03:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1145592 >> Mar 18 16:03:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1145592 >> Mar 18 16:03:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1145504 >> Mar 18 16:03:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1145504 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1146780 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1146780 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1215552 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1215552 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1215596 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1215596 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1192760 >> Mar 18 16:03:49 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1192760 >> Mar 18 16:03:52 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1168956 >> Mar 18 16:03:52 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1168956 >> Mar 18 16:04:01 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1191924 >> Mar 18 16:04:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1191924 >> Mar 18 16:04:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1214276 >> Mar 18 16:04:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1214276 >> Mar 18 16:04:09 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1204948 >> Mar 18 16:04:09 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1204948 >> Mar 18 16:04:12 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1214584 >> Mar 18 16:04:12 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1214584 >> Mar 18 16:04:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1215200 >> Mar 18 16:04:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1215200 >> Mar 18 16:04:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1216300 >> Mar 18 16:04:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1216300 >> Mar 18 16:05:01 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1217356 >> Mar 18 16:05:01 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1217356 >> Mar 18 16:05:07 storage kernel: qla2xxx/50:01:43:80:06:37:c4:14: Unsupported >> SCSI Opcode 0x85, sending CHECK_CONDITION. >> Mar 18 16:05:21 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1218456 >> Mar 18 16:05:21 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1218456 >> Mar 18 16:05:28 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1218852 >> Mar 18 16:05:28 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1218852 >> Mar 18 16:05:41 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1219424 >> Mar 18 16:05:41 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1219424 >> Mar 18 16:05:47 storage kernel: ABORT_TASK: Found referenced qla2xxx >> task_tag: 1220084 >> Mar 18 16:05:47 storage kernel: ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST >> for ref_tag: 1220084 >> >> >> I have grepped out the NON_EXISTENT_LUN errors, as those are expected and >> would make it more difficult to see the relevant logs. >> >> Please let me know if I can do anything else to help out. >> >> Thanks! >> David >> >> On Thu, Mar 17, 2016 at 1:23 PM, Dan Lane <dracodan@xxxxxxxxx> wrote: >>> >>> On Wed, Mar 16, 2016 at 5:08 PM, Dan Lane <dracodan@xxxxxxxxx> wrote: >>> > On Wed, Mar 16, 2016 at 10:47 AM, Dan Lane <dracodan@xxxxxxxxx> wrote: >>> >> On Tue, Mar 15, 2016 at 3:49 PM, Dan Lane <dracodan@xxxxxxxxx> wrote: >>> >>> On Tue, Mar 15, 2016 at 3:45 PM, Dan Lane <dracodan@xxxxxxxxx> wrote: >>> >>>> I went to pull the latest source and I noticed mainline Kernel 4.5 >>> >>>> was >>> >>>> released yesterday. Did all the recent patches that apply to fiber >>> >>>> channel make it into this release or do I still need to patch? >>> >>>> >>> >>>> Thanks >>> >>>> Dan >>> >>>> >>> >>>> On Fri, Mar 11, 2016 at 11:07 PM, Nicholas A. Bellinger >>> >>>> <nab@xxxxxxxxxxxxxxx> wrote: >>> >>>>> On Fri, 2016-03-11 at 18:15 -0500, Dan Lane wrote: >>> >>>>>> I'm back in town now and ready to try this again. Should I still >>> >>>>>> try >>> >>>>>> this patch? >>> >>>>> >>> >>>>> Yes, you still need to apply the patch to drop the extra bogus >>> >>>>> target_put_sess_cmd() call, when !__target_check_io_state() for >>> >>>>> ABORT_TASK occurs: >>> >>>>> >>> >>>>> >>> >>>>> https://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?id=7f54ab5ff52fb0b91569bc69c4a6bc5cac1b768d >>> >>>>> >>> >>>>>> I noticed you had submitted a patch a few days ago, so >>> >>>>>> can I just pull all the latest updates from your git repo? >>> >>>>>> >>> >>>>> >>> >>>>> The PULL request just went out to Linus, and will be included for >>> >>>>> v4.5 release. >>> >>>>> >>> >>>>> It's also CC'ed for stable, and will make it's way down to v3.14.y >>> >>>>> stable over the next weeks. >>> >>>>> >>> >>> >>> >>> Whoops, sorry about the top post... I know you said the request was >>> >>> sent to Linus, I just wanted to confirm that it made it since the time >>> >>> frame between that last email and when 4.5 was released was so short. >>> >>> >>> >>> Thanks again >>> >> >>> >> Latest update: >>> >> With the 4.5 (final) kernel my storage was more stable than ever, but >>> >> again went inaccessible after about 15 hours. This is despite very >>> >> heavy usage last night, the likes of which caused failures in the past >>> >> (but I was amazed with the performance, I was able to get 650MB/s >>> >> writes and 750MB/s reads!!!). The aborts seem to be coming in as >>> >> steady as they have in the past, which leads me to believe the patch >>> >> for the extra "bogus target_put_sess_cmd() call" didn't make it in >>> >> time for the 4.5 release. If it did, this means there are more >>> >> problems. >>> >> >>> >> Here is a snippet from my messages log before ESXi gave up: >>> >> Mar 16 07:21:57 dracofiler kernel: ABORT_TASK: Sending >>> >> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1169660 >>> >> Mar 16 07:21:57 dracofiler kernel: ABORT_TASK: Found referenced >>> >> qla2xxx task_tag: 1169616 >>> >> Mar 16 07:21:57 dracofiler kernel: ABORT_TASK: Sending >>> >> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1169616 >>> >> Mar 16 07:23:20 dracofiler kernel: ABORT_TASK: Found referenced >>> >> qla2xxx task_tag: 1147132 >>> >> Mar 16 07:23:20 dracofiler kernel: ABORT_TASK: Sending >>> >> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1147132 >>> >> Mar 16 07:23:20 dracofiler kernel: ABORT_TASK: Found referenced >>> >> qla2xxx task_tag: 1147176 >>> >> Mar 16 07:23:24 dracofiler kernel: ABORT_TASK: Sending >>> >> TMR_FUNCTION_COMPLETE for ref_tag: 1147176 >>> >> Mar 16 07:23:24 dracofiler kernel: ABORT_TASK: Found referenced >>> >> qla2xxx task_tag: 1186556 >>> >> >>> >> Also, I configured my hosts to send their logs to a syslog server, I >>> >> have an appointment to go to but I'll pull those and send them to you >>> >> this afternoon. >>> >> >>> >> Thanks >>> >> Dan >>> > >>> > I discovered the ATS heartbeat issue was still causing issues. I have >>> > created a host profile and applied it to all of my hosts to ensure it >>> > doesn't come up again. For now there's no reason to dig further with >>> > this, I will report back whether or not I'm still having the issue in >>> > the next few days (or sooner if it still fails). >>> > >>> > Thanks, >>> > Dan >>> >>> >>> Okay, back on track, I'm still seeing these aborts and eventually >>> losing access to the storage despite running the final 4.5 kernel and >>> VMFS3.UseATSForHBonVMFS5=0 set on all hosts. According to the log and >>> what you have explained in the past, I think it still looks like I'm >>> using ATS heartbeat, but I may be wrong. Note, it takes a lot longer >>> to fail than it used to, but I can still trigger the failure by >>> running ATTO repeatedly from a VM. Also, for comparing between the >>> logs, my timezone is GMT/Zulu -4 (target server is local time, >>> vmkernel.log is zulu). >>> >>> Here is my target log from the time period when it finally failed >>> (atto was running from a VM at this time): >>> Mar 16 23:30:49 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1165392 >>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1165744 >>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1165744 >>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1165832 >>> Mar 16 23:30:51 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1165832 >>> Mar 16 23:31:06 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1180616 >>> Mar 16 23:31:06 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1180616 >>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1182244 >>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182244 >>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1182200 >>> Mar 16 23:31:09 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182200 >>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1182552 >>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182552 >>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1182508 >>> Mar 16 23:31:11 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1182508 >>> Mar 16 23:34:18 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1152236 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_FUNCTION_COMPLETE for ref_tag: 1152236 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1161124 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1161124 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1199888 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1199888 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1156680 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1166976 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1168164 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1168164 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1156680 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1156680 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1172784 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1134064 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1134064 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1174720 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1174720 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1134152 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1134152 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1215288 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1185368 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1185632 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1223648 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1192012 >>> Mar 16 23:34:28 dracofiler kernel: Detected MISCOMPARE for addr: >>> ffff88062f84c000 buf: ffff88062d360c00 >>> Mar 16 23:34:28 dracofiler kernel: Target/iblock: Send MISCOMPARE >>> check condition and sense >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_FUNCTION_COMPLETE for ref_tag: 1192012 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1201912 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1143128 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1176524 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1172476 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1135824 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1169044 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1173136 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1174060 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1198084 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1174192 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1187084 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1146560 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1146604 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1196148 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1152280 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1144844 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1145768 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1146120 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1191748 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1159496 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1159540 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1170848 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1219336 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1219424 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1179692 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1208600 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186028 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186160 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186204 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1186248 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1193508 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1200548 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1151796 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1155712 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1185852 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1200284 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1219028 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1139300 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1150168 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1150212 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1136000 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Sending >>> TMR_TASK_DOES_NOT_EXIST for ref_tag: 1136000 >>> Mar 16 23:34:28 dracofiler kernel: ABORT_TASK: Found referenced >>> qla2xxx task_tag: 1218896 >>> >>> >>> Dan >> >> > > So.... is someone looking into this? Is there anything else we can > contribute to help? > > Thanks > Dan Nicholas, So... have you just thrown in the towel on this? Can we at least get an "I'm looking at it" response? Dan -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
