Hi Nicholas,
I think we're a little closer to root cause on this. Below are the first /ver/log/messages error with a text dump of the Wireshark packet. At least the DataDigests match if you reverse order
one of them (0x2b5d89ff -LIO versus 0xff895d2b-Wireshark). The Wireshark dissector is saying the DataDigest as transmitted by the initiator is good. The odd offset you noted yesterday may be behind this.
Thanks,
-Joe
Error in /var/log/messages:
ITT: 0x00170118, Offset: 6048, Length: 8192, DataSN: 0x00000000, CRC32C DataDigest 0x2b5d89ff does not match computed 0x18d66934
Wireshark:
No. Time Source Destination Protocol Length Info
58526 147.042348 172.30.11.70 172.30.11.78 iSCSI 1170 SCSI: Data Out LUN: 0x00 (Write(10) Request Data)
Frame 58526: 1170 bytes on wire (9360 bits), 1170 bytes captured (9360 bits)
Arrival Time: Dec 22, 2011 11:00:05.708437000 PST
Epoch Time: 1324580405.708437000 seconds
[Time delta from previous captured frame: 0.000003000 seconds]
[Time delta from previous displayed frame: 0.000105000 seconds]
[Time since reference or first frame: 147.042348000 seconds]
Frame Number: 58526
Frame Length: 1170 bytes (9360 bits)
Capture Length: 1170 bytes (9360 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ip:tcp:iscsi:data]
[Coloring Rule Name: TCP]
[Coloring Rule String: tcp]
Ethernet II, Src: Serveren_07:59:81 (00:16:88:07:59:81), Dst: Vmware_91:00:27 (00:50:56:91:00:27)
Destination: Vmware_91:00:27 (00:50:56:91:00:27)
Address: Vmware_91:00:27 (00:50:56:91:00:27)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
Source: Serveren_07:59:81 (00:16:88:07:59:81)
Address: Serveren_07:59:81 (00:16:88:07:59:81)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
Type: IP (0x0800)
Internet Protocol Version 4, Src: 172.30.11.70 (172.30.11.70), Dst: 172.30.11.78 (172.30.11.78)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: 1156
Identification: 0x9091 (37009)
Flags: 0x00
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: TCP (6)
Header checksum: 0x7712 [correct]
[Good: True]
[Bad: False]
Source: 172.30.11.70 (172.30.11.70)
Destination: 172.30.11.78 (172.30.11.78)
Transmission Control Protocol, Src Port: 22699 (22699), Dst Port: iscsi-target (3260), Seq: 52752545, Ack: 52633537, Len: 1104
Source port: 22699 (22699)
Destination port: iscsi-target (3260)
[Stream index: 1]
Sequence number: 52752545 (relative sequence number)
[Next sequence number: 52753649 (relative sequence number)]
Acknowledgement number: 52633537 (relative ack number)
Header length: 32 bytes
Flags: 0x18 (PSH, ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Nonce: Not set
.... 0... .... = Congestion Window Reduced (CWR): Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgement: Set
.... .... 1... = Push: Set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
Window size value: 65535
[Calculated window size: 65535]
[Window size scaling factor: -2 (no window scaling used)]
Checksum: 0xeb1b [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
Options: (12 bytes)
No-Operation (NOP)
No-Operation (NOP)
Timestamps: TSval 7943405, TSecr 126790
Kind: Timestamp (8)
Length: 10
Timestamp value: 7943405
Timestamp echo reply: 126790
[SEQ/ACK analysis]
[Bytes in flight: 8248]
TCP segment data (1104 bytes)
[6 Reassembled TCP Segments (8248 bytes): #58521(1432), #58522(1428), #58523(1428), #58524(1428), #58525(1428), #58526(1104)]
[Frame: 58521, payload: 0-1431 (1432 bytes)]
[Frame: 58522, payload: 1432-2859 (1428 bytes)]
[Frame: 58523, payload: 2860-4287 (1428 bytes)]
[Frame: 58524, payload: 4288-5715 (1428 bytes)]
[Frame: 58525, payload: 5716-7143 (1428 bytes)]
[Frame: 58526, payload: 7144-8247 (1104 bytes)]
[Segment count: 6]
[Reassembled TCP length: 8248]
iSCSI (SCSI Data Out)
Opcode: SCSI Data Out (0x05)
Flags: 0x00
0... .... = F: Not final PDU in sequence
TotalAHSLength: 0x00
DataSegmentLength: 0x00002000
LUN: 0000000000000000
InitiatorTaskTag: 0x00170118
TargetTransferTag: 0x00000a14
ExpStatSN: 0xb3beaa8f
DataSN: 0x00000000
BufferOffset: 0x000017a0
HeaderDigest: 0xd34ac92d (Good CRC32)
DataDigest: 0xff895d2b (Good CRC32)
Request in: 58499
SCSI Payload (Write(10) Request Data)
[LUN: 0x0000]
[Command Set:Direct Access Device (0x00) (Using default commandset)]
[SBC Opcode: Write(10) (0x2a)]
[Request in: 58499]
Data (8192 bytes)
0000 d0 0b d1 0b d2 0b d3 0b d4 0b d5 0b d6 0b d7 0b ................
0010 d8 0b d9 0b da 0b db 0b dc 0b dd 0b de 0b df 0b ................
0020 e0 0b e1 0b e2 0b e3 0b e4 0b e5 0b e6 0b e7 0b ................
0030 e8 0b e9 0b ea 0b eb 0b ec 0b ed 0b ee 0b ef 0b ................
0040 f0 0b f1 0b f2 0b f3 0b f4 0b f5 0b f6 0b f7 0b ................
0050 f8 0b f9 0b fa 0b fb 0b fc 0b fd 0b fe 0b ff 0b ................
0060 00 0c 01 0c 02 0c 03 0c 04 0c 05 0c 06 0c 07 0c ................
0070 08 0c 09 0c 0a 0c 0b 0c 0c 0c 0d 0c 0e 0c 0f 0c ................
0080 10 0c 11 0c 12 0c 13 0c 14 0c 15 0c 16 0c 17 0c ................
0090 18 0c 19 0c 1a 0c 1b 0c 1c 0c 1d 0c 1e 0c 1f 0c ................
00a0 20 0c 21 0c 22 0c 23 0c 24 0c 25 0c 26 0c 27 0c .!.".#.$.%.&.'.
00b0 28 0c 29 0c 2a 0c 2b 0c 2c 0c 2d 0c 2e 0c 2f 0c (.).*.+.,.-.../.
00c0 30 0c 31 0c 32 0c 33 0c 34 0c 35 0c 36 0c 37 0c 0.1.2.3.4.5.6.7.
00d0 38 0c 39 0c 3a 0c 3b 0c 3c 0c 3d 0c 3e 0c 3f 0c 8.9.:.;.<.=.>.?.
00e0 40 0c 41 0c 42 0c 43 0c 44 0c 45 0c 46 0c 47 0c @.A.B.C.D.E.F.G.
00f0 48 0c 49 0c 4a 0c 4b 0c 4c 0c 4d 0c 4e 0c 4f 0c H.I.J.K.L.M.N.O.
0100 50 0c 51 0c 52 0c 53 0c 54 0c 55 0c 56 0c 57 0c P.Q.R.S.T.U.V.W.
0110 58 0c 59 0c 5a 0c 5b 0c 5c 0c 5d 0c 5e 0c 5f 0c X.Y.Z.[.\.].^._.
0120 60 0c 61 0c 62 0c 63 0c 64 0c 65 0c 66 0c 67 0c `.a.b.c.d.e.f.g.
0130 68 0c 69 0c 6a 0c 6b 0c 6c 0c 6d 0c 6e 0c 6f 0c h.i.j.k.l.m.n.o.
0140 70 0c 71 0c 72 0c 73 0c 74 0c 75 0c 76 0c 77 0c p.q.r.s.t.u.v.w.
0150 78 0c 79 0c 7a 0c 7b 0c 7c 0c 7d 0c 7e 0c 7f 0c x.y.z.{.|.}.~...
0160 80 0c 81 0c 82 0c 83 0c 84 0c 85 0c 86 0c 87 0c ................
0170 88 0c 89 0c 8a 0c 8b 0c 8c 0c 8d 0c 8e 0c 8f 0c ................
0180 90 0c 91 0c 92 0c 93 0c 94 0c 95 0c 96 0c 97 0c ................
0190 98 0c 99 0c 9a 0c 9b 0c 9c 0c 9d 0c 9e 0c 9f 0c ................
01a0 a0 0c a1 0c a2 0c a3 0c a4 0c a5 0c a6 0c a7 0c ................
01b0 a8 0c a9 0c aa 0c ab 0c ac 0c ad 0c ae 0c af 0c ................
01c0 b0 0c b1 0c b2 0c b3 0c b4 0c b5 0c b6 0c b7 0c ................
01d0 b8 0c b9 0c ba 0c bb 0c bc 0c bd 0c be 0c bf 0c ................
01e0 c0 0c c1 0c c2 0c c3 0c c4 0c c5 0c c6 0c c7 0c ................
01f0 c8 0c c9 0c ca 0c cb 0c cc 0c cd 0c ce 0c cf 0c ................
0200 d0 0c d1 0c d2 0c d3 0c d4 0c d5 0c d6 0c d7 0c ................
0210 d8 0c d9 0c da 0c db 0c dc 0c dd 0c de 0c df 0c ................
0220 e0 0c e1 0c e2 0c e3 0c e4 0c e5 0c e6 0c e7 0c ................
0230 e8 0c e9 0c ea 0c eb 0c ec 0c ed 0c ee 0c ef 0c ................
0240 f0 0c f1 0c f2 0c f3 0c f4 0c f5 0c f6 0c f7 0c ................
0250 f8 0c f9 0c fa 0c fb 0c fc 0c fd 0c fe 0c ff 0c ................
0260 00 0d 01 0d 02 0d 03 0d 04 0d 05 0d 06 0d 07 0d ................
0270 08 0d 09 0d 0a 0d 0b 0d 0c 0d 0d 0d 0e 0d 0f 0d ................
0280 10 0d 11 0d 12 0d 13 0d 14 0d 15 0d 16 0d 17 0d ................
0290 18 0d 19 0d 1a 0d 1b 0d 1c 0d 1d 0d 1e 0d 1f 0d ................
02a0 20 0d 21 0d 22 0d 23 0d 24 0d 25 0d 26 0d 27 0d .!.".#.$.%.&.'.
02b0 28 0d 29 0d 2a 0d 2b 0d 2c 0d 2d 0d 2e 0d 2f 0d (.).*.+.,.-.../.
02c0 30 0d 31 0d 32 0d 33 0d 34 0d 35 0d 36 0d 37 0d 0.1.2.3.4.5.6.7.
02d0 38 0d 39 0d 3a 0d 3b 0d 3c 0d 3d 0d 3e 0d 3f 0d 8.9.:.;.<.=.>.?.
02e0 40 0d 41 0d 42 0d 43 0d 44 0d 45 0d 46 0d 47 0d @.A.B.C.D.E.F.G.
02f0 48 0d 49 0d 4a 0d 4b 0d 4c 0d 4d 0d 4e 0d 4f 0d H.I.J.K.L.M.N.O.
0300 50 0d 51 0d 52 0d 53 0d 54 0d 55 0d 56 0d 57 0d P.Q.R.S.T.U.V.W.
0310 58 0d 59 0d 5a 0d 5b 0d 5c 0d 5d 0d 5e 0d 5f 0d X.Y.Z.[.\.].^._.
0320 60 0d 61 0d 62 0d 63 0d 64 0d 65 0d 66 0d 67 0d `.a.b.c.d.e.f.g.
0330 68 0d 69 0d 6a 0d 6b 0d 6c 0d 6d 0d 6e 0d 6f 0d h.i.j.k.l.m.n.o.
0340 70 0d 71 0d 72 0d 73 0d 74 0d 75 0d 76 0d 77 0d p.q.r.s.t.u.v.w.
0350 78 0d 79 0d 7a 0d 7b 0d 7c 0d 7d 0d 7e 0d 7f 0d x.y.z.{.|.}.~...
0360 80 0d 81 0d 82 0d 83 0d 84 0d 85 0d 86 0d 87 0d ................
0370 88 0d 89 0d 8a 0d 8b 0d 8c 0d 8d 0d 8e 0d 8f 0d ................
0380 90 0d 91 0d 92 0d 93 0d 94 0d 95 0d 96 0d 97 0d ................
0390 98 0d 99 0d 9a 0d 9b 0d 9c 0d 9d 0d 9e 0d 9f 0d ................
03a0 a0 0d a1 0d a2 0d a3 0d a4 0d a5 0d a6 0d a7 0d ................
03b0 a8 0d a9 0d aa 0d ab 0d ac 0d ad 0d ae 0d af 0d ................
03c0 b0 0d b1 0d b2 0d b3 0d b4 0d b5 0d b6 0d b7 0d ................
03d0 b8 0d b9 0d ba 0d bb 0d bc 0d bd 0d be 0d bf 0d ................
03e0 c0 0d c1 0d c2 0d c3 0d c4 0d c5 0d c6 0d c7 0d ................
03f0 c8 0d c9 0d ca 0d cb 0d cc 0d cd 0d ce 0d cf 0d ................
0400 d0 0d d1 0d d2 0d d3 0d d4 0d d5 0d d6 0d d7 0d ................
0410 d8 0d d9 0d da 0d db 0d dc 0d dd 0d de 0d df 0d ................
0420 e0 0d e1 0d e2 0d e3 0d e4 0d e5 0d e6 0d e7 0d ................
0430 e8 0d e9 0d ea 0d eb 0d ec 0d ed 0d ee 0d ef 0d ................
0440 f0 0d f1 0d f2 0d f3 0d f4 0d f5 0d f6 0d f7 0d ................
0450 f8 0d f9 0d fa 0d fb 0d fc 0d fd 0d fe 0d ff 0d ................
0460 00 0e 01 0e 02 0e 03 0e 04 0e 05 0e 06 0e 07 0e ................
0470 08 0e 09 0e 0a 0e 0b 0e 0c 0e 0d 0e 0e 0e 0f 0e ................
0480 10 0e 11 0e 12 0e 13 0e 14 0e 15 0e 16 0e 17 0e ................
0490 18 0e 19 0e 1a 0e 1b 0e 1c 0e 1d 0e 1e 0e 1f 0e ................
04a0 20 0e 21 0e 22 0e 23 0e 24 0e 25 0e 26 0e 27 0e .!.".#.$.%.&.'.
04b0 28 0e 29 0e 2a 0e 2b 0e 2c 0e 2d 0e 2e 0e 2f 0e (.).*.+.,.-.../.
04c0 30 0e 31 0e 32 0e 33 0e 34 0e 35 0e 36 0e 37 0e 0.1.2.3.4.5.6.7.
04d0 38 0e 39 0e 3a 0e 3b 0e 3c 0e 3d 0e 3e 0e 3f 0e 8.9.:.;.<.=.>.?.
04e0 40 0e 41 0e 42 0e 43 0e 44 0e 45 0e 46 0e 47 0e @.A.B.C.D.E.F.G.
04f0 48 0e 49 0e 4a 0e 4b 0e 4c 0e 4d 0e 4e 0e 4f 0e H.I.J.K.L.M.N.O.
0500 50 0e 51 0e 52 0e 53 0e 54 0e 55 0e 56 0e 57 0e P.Q.R.S.T.U.V.W.
0510 58 0e 59 0e 5a 0e 5b 0e 5c 0e 5d 0e 5e 0e 5f 0e X.Y.Z.[.\.].^._.
0520 60 0e 61 0e 62 0e 63 0e 64 0e 65 0e 66 0e 67 0e `.a.b.c.d.e.f.g.
0530 68 0e 69 0e 6a 0e 6b 0e 6c 0e 6d 0e 6e 0e 6f 0e h.i.j.k.l.m.n.o.
0540 70 0e 71 0e 72 0e 73 0e 74 0e 75 0e 76 0e 77 0e p.q.r.s.t.u.v.w.
0550 78 0e 79 0e 7a 0e 7b 0e 7c 0e 7d 0e 7e 0e 7f 0e x.y.z.{.|.}.~...
0560 80 0e 81 0e 82 0e 83 0e 84 0e 85 0e 86 0e 87 0e ................
0570 88 0e 89 0e 8a 0e 8b 0e 8c 0e 8d 0e 8e 0e 8f 0e ................
0580 90 0e 91 0e 92 0e 93 0e 94 0e 95 0e 96 0e 97 0e ................
0590 98 0e 99 0e 9a 0e 9b 0e 9c 0e 9d 0e 9e 0e 9f 0e ................
05a0 a0 0e a1 0e a2 0e a3 0e a4 0e a5 0e a6 0e a7 0e ................
05b0 a8 0e a9 0e aa 0e ab 0e ac 0e ad 0e ae 0e af 0e ................
05c0 b0 0e b1 0e b2 0e b3 0e b4 0e b5 0e b6 0e b7 0e ................
05d0 b8 0e b9 0e ba 0e bb 0e bc 0e bd 0e be 0e bf 0e ................
05e0 c0 0e c1 0e c2 0e c3 0e c4 0e c5 0e c6 0e c7 0e ................
05f0 c8 0e c9 0e ca 0e cb 0e cc 0e cd 0e ce 0e cf 0e ................
0600 d0 0e d1 0e d2 0e d3 0e d4 0e d5 0e d6 0e d7 0e ................
0610 d8 0e d9 0e da 0e db 0e dc 0e dd 0e de 0e df 0e ................
0620 e0 0e e1 0e e2 0e e3 0e e4 0e e5 0e e6 0e e7 0e ................
0630 e8 0e e9 0e ea 0e eb 0e ec 0e ed 0e ee 0e ef 0e ................
0640 f0 0e f1 0e f2 0e f3 0e f4 0e f5 0e f6 0e f7 0e ................
0650 f8 0e f9 0e fa 0e fb 0e fc 0e fd 0e fe 0e ff 0e ................
0660 00 0f 01 0f 02 0f 03 0f 04 0f 05 0f 06 0f 07 0f ................
0670 08 0f 09 0f 0a 0f 0b 0f 0c 0f 0d 0f 0e 0f 0f 0f ................
0680 10 0f 11 0f 12 0f 13 0f 14 0f 15 0f 16 0f 17 0f ................
0690 18 0f 19 0f 1a 0f 1b 0f 1c 0f 1d 0f 1e 0f 1f 0f ................
06a0 20 0f 21 0f 22 0f 23 0f 24 0f 25 0f 26 0f 27 0f .!.".#.$.%.&.'.
06b0 28 0f 29 0f 2a 0f 2b 0f 2c 0f 2d 0f 2e 0f 2f 0f (.).*.+.,.-.../.
06c0 30 0f 31 0f 32 0f 33 0f 34 0f 35 0f 36 0f 37 0f 0.1.2.3.4.5.6.7.
06d0 38 0f 39 0f 3a 0f 3b 0f 3c 0f 3d 0f 3e 0f 3f 0f 8.9.:.;.<.=.>.?.
06e0 40 0f 41 0f 42 0f 43 0f 44 0f 45 0f 46 0f 47 0f @.A.B.C.D.E.F.G.
06f0 48 0f 49 0f 4a 0f 4b 0f 4c 0f 4d 0f 4e 0f 4f 0f H.I.J.K.L.M.N.O.
0700 50 0f 51 0f 52 0f 53 0f 54 0f 55 0f 56 0f 57 0f P.Q.R.S.T.U.V.W.
0710 58 0f 59 0f 5a 0f 5b 0f 5c 0f 5d 0f 5e 0f 5f 0f X.Y.Z.[.\.].^._.
0720 60 0f 61 0f 62 0f 63 0f 64 0f 65 0f 66 0f 67 0f `.a.b.c.d.e.f.g.
0730 68 0f 69 0f 6a 0f 6b 0f 6c 0f 6d 0f 6e 0f 6f 0f h.i.j.k.l.m.n.o.
0740 70 0f 71 0f 72 0f 73 0f 74 0f 75 0f 76 0f 77 0f p.q.r.s.t.u.v.w.
0750 78 0f 79 0f 7a 0f 7b 0f 7c 0f 7d 0f 7e 0f 7f 0f x.y.z.{.|.}.~...
0760 80 0f 81 0f 82 0f 83 0f 84 0f 85 0f 86 0f 87 0f ................
0770 88 0f 89 0f 8a 0f 8b 0f 8c 0f 8d 0f 8e 0f 8f 0f ................
0780 90 0f 91 0f 92 0f 93 0f 94 0f 95 0f 96 0f 97 0f ................
0790 98 0f 99 0f 9a 0f 9b 0f 9c 0f 9d 0f 9e 0f 9f 0f ................
07a0 a0 0f a1 0f a2 0f a3 0f a4 0f a5 0f a6 0f a7 0f ................
07b0 a8 0f a9 0f aa 0f ab 0f ac 0f ad 0f ae 0f af 0f ................
07c0 b0 0f b1 0f b2 0f b3 0f b4 0f b5 0f b6 0f b7 0f ................
07d0 b8 0f b9 0f ba 0f bb 0f bc 0f bd 0f be 0f bf 0f ................
07e0 c0 0f c1 0f c2 0f c3 0f c4 0f c5 0f c6 0f c7 0f ................
07f0 c8 0f c9 0f ca 0f cb 0f cc 0f cd 0f ce 0f cf 0f ................
0800 d0 0f d1 0f d2 0f d3 0f d4 0f d5 0f d6 0f d7 0f ................
0810 d8 0f d9 0f da 0f db 0f dc 0f dd 0f de 0f df 0f ................
0820 e0 0f e1 0f e2 0f e3 0f e4 0f e5 0f e6 0f e7 0f ................
0830 e8 0f e9 0f ea 0f eb 0f ec 0f ed 0f ee 0f ef 0f ................
0840 f0 0f f1 0f f2 0f f3 0f f4 0f f5 0f f6 0f f7 0f ................
0850 f8 0f f9 0f fa 0f fb 0f fc 0f fd 0f fe 0f ff 0f ................
0860 00 10 01 10 02 10 03 10 04 10 05 10 06 10 07 10 ................
0870 08 10 09 10 0a 10 0b 10 0c 10 0d 10 0e 10 0f 10 ................
0880 10 10 11 10 12 10 13 10 14 10 15 10 16 10 17 10 ................
0890 18 10 19 10 1a 10 1b 10 1c 10 1d 10 1e 10 1f 10 ................
08a0 20 10 21 10 22 10 23 10 24 10 25 10 26 10 27 10 .!.".#.$.%.&.'.
08b0 28 10 29 10 2a 10 2b 10 2c 10 2d 10 2e 10 2f 10 (.).*.+.,.-.../.
08c0 30 10 31 10 32 10 33 10 34 10 35 10 36 10 37 10 0.1.2.3.4.5.6.7.
08d0 38 10 39 10 3a 10 3b 10 3c 10 3d 10 3e 10 3f 10 8.9.:.;.<.=.>.?.
08e0 40 10 41 10 42 10 43 10 44 10 45 10 46 10 47 10 @.A.B.C.D.E.F.G.
08f0 48 10 49 10 4a 10 4b 10 4c 10 4d 10 4e 10 4f 10 H.I.J.K.L.M.N.O.
0900 50 10 51 10 52 10 53 10 54 10 55 10 56 10 57 10 P.Q.R.S.T.U.V.W.
0910 58 10 59 10 5a 10 5b 10 5c 10 5d 10 5e 10 5f 10 X.Y.Z.[.\.].^._.
0920 60 10 61 10 62 10 63 10 64 10 65 10 66 10 67 10 `.a.b.c.d.e.f.g.
0930 68 10 69 10 6a 10 6b 10 6c 10 6d 10 6e 10 6f 10 h.i.j.k.l.m.n.o.
0940 70 10 71 10 72 10 73 10 74 10 75 10 76 10 77 10 p.q.r.s.t.u.v.w.
0950 78 10 79 10 7a 10 7b 10 7c 10 7d 10 7e 10 7f 10 x.y.z.{.|.}.~...
0960 80 10 81 10 82 10 83 10 84 10 85 10 86 10 87 10 ................
0970 88 10 89 10 8a 10 8b 10 8c 10 8d 10 8e 10 8f 10 ................
0980 90 10 91 10 92 10 93 10 94 10 95 10 96 10 97 10 ................
0990 98 10 99 10 9a 10 9b 10 9c 10 9d 10 9e 10 9f 10 ................
09a0 a0 10 a1 10 a2 10 a3 10 a4 10 a5 10 a6 10 a7 10 ................
09b0 a8 10 a9 10 aa 10 ab 10 ac 10 ad 10 ae 10 af 10 ................
09c0 b0 10 b1 10 b2 10 b3 10 b4 10 b5 10 b6 10 b7 10 ................
09d0 b8 10 b9 10 ba 10 bb 10 bc 10 bd 10 be 10 bf 10 ................
09e0 c0 10 c1 10 c2 10 c3 10 c4 10 c5 10 c6 10 c7 10 ................
09f0 c8 10 c9 10 ca 10 cb 10 cc 10 cd 10 ce 10 cf 10 ................
0a00 d0 10 d1 10 d2 10 d3 10 d4 10 d5 10 d6 10 d7 10 ................
0a10 d8 10 d9 10 da 10 db 10 dc 10 dd 10 de 10 df 10 ................
0a20 e0 10 e1 10 e2 10 e3 10 e4 10 e5 10 e6 10 e7 10 ................
0a30 e8 10 e9 10 ea 10 eb 10 ec 10 ed 10 ee 10 ef 10 ................
0a40 f0 10 f1 10 f2 10 f3 10 f4 10 f5 10 f6 10 f7 10 ................
0a50 f8 10 f9 10 fa 10 fb 10 fc 10 fd 10 fe 10 ff 10 ................
0a60 00 11 01 11 02 11 03 11 04 11 05 11 06 11 07 11 ................
0a70 08 11 09 11 0a 11 0b 11 0c 11 0d 11 0e 11 0f 11 ................
0a80 10 11 11 11 12 11 13 11 14 11 15 11 16 11 17 11 ................
0a90 18 11 19 11 1a 11 1b 11 1c 11 1d 11 1e 11 1f 11 ................
0aa0 20 11 21 11 22 11 23 11 24 11 25 11 26 11 27 11 .!.".#.$.%.&.'.
0ab0 28 11 29 11 2a 11 2b 11 2c 11 2d 11 2e 11 2f 11 (.).*.+.,.-.../.
0ac0 30 11 31 11 32 11 33 11 34 11 35 11 36 11 37 11 0.1.2.3.4.5.6.7.
0ad0 38 11 39 11 3a 11 3b 11 3c 11 3d 11 3e 11 3f 11 8.9.:.;.<.=.>.?.
0ae0 40 11 41 11 42 11 43 11 44 11 45 11 46 11 47 11 @.A.B.C.D.E.F.G.
0af0 48 11 49 11 4a 11 4b 11 4c 11 4d 11 4e 11 4f 11 H.I.J.K.L.M.N.O.
0b00 50 11 51 11 52 11 53 11 54 11 55 11 56 11 57 11 P.Q.R.S.T.U.V.W.
0b10 58 11 59 11 5a 11 5b 11 5c 11 5d 11 5e 11 5f 11 X.Y.Z.[.\.].^._.
0b20 60 11 61 11 62 11 63 11 64 11 65 11 66 11 67 11 `.a.b.c.d.e.f.g.
0b30 68 11 69 11 6a 11 6b 11 6c 11 6d 11 6e 11 6f 11 h.i.j.k.l.m.n.o.
0b40 70 11 71 11 72 11 73 11 74 11 75 11 76 11 77 11 p.q.r.s.t.u.v.w.
0b50 78 11 79 11 7a 11 7b 11 7c 11 7d 11 7e 11 7f 11 x.y.z.{.|.}.~...
0b60 80 11 81 11 82 11 83 11 84 11 85 11 86 11 87 11 ................
0b70 88 11 89 11 8a 11 8b 11 8c 11 8d 11 8e 11 8f 11 ................
0b80 90 11 91 11 92 11 93 11 94 11 95 11 96 11 97 11 ................
0b90 98 11 99 11 9a 11 9b 11 9c 11 9d 11 9e 11 9f 11 ................
0ba0 a0 11 a1 11 a2 11 a3 11 a4 11 a5 11 a6 11 a7 11 ................
0bb0 a8 11 a9 11 aa 11 ab 11 ac 11 ad 11 ae 11 af 11 ................
0bc0 b0 11 b1 11 b2 11 b3 11 b4 11 b5 11 b6 11 b7 11 ................
0bd0 b8 11 b9 11 ba 11 bb 11 bc 11 bd 11 be 11 bf 11 ................
0be0 c0 11 c1 11 c2 11 c3 11 c4 11 c5 11 c6 11 c7 11 ................
0bf0 c8 11 c9 11 ca 11 cb 11 cc 11 cd 11 ce 11 cf 11 ................
0c00 d0 11 d1 11 d2 11 d3 11 d4 11 d5 11 d6 11 d7 11 ................
0c10 d8 11 d9 11 da 11 db 11 dc 11 dd 11 de 11 df 11 ................
0c20 e0 11 e1 11 e2 11 e3 11 e4 11 e5 11 e6 11 e7 11 ................
0c30 e8 11 e9 11 ea 11 eb 11 ec 11 ed 11 ee 11 ef 11 ................
0c40 f0 11 f1 11 f2 11 f3 11 f4 11 f5 11 f6 11 f7 11 ................
0c50 f8 11 f9 11 fa 11 fb 11 fc 11 fd 11 fe 11 ff 11 ................
0c60 00 12 01 12 02 12 03 12 04 12 05 12 06 12 07 12 ................
0c70 08 12 09 12 0a 12 0b 12 0c 12 0d 12 0e 12 0f 12 ................
0c80 10 12 11 12 12 12 13 12 14 12 15 12 16 12 17 12 ................
0c90 18 12 19 12 1a 12 1b 12 1c 12 1d 12 1e 12 1f 12 ................
0ca0 20 12 21 12 22 12 23 12 24 12 25 12 26 12 27 12 .!.".#.$.%.&.'.
0cb0 28 12 29 12 2a 12 2b 12 2c 12 2d 12 2e 12 2f 12 (.).*.+.,.-.../.
0cc0 30 12 31 12 32 12 33 12 34 12 35 12 36 12 37 12 0.1.2.3.4.5.6.7.
0cd0 38 12 39 12 3a 12 3b 12 3c 12 3d 12 3e 12 3f 12 8.9.:.;.<.=.>.?.
0ce0 40 12 41 12 42 12 43 12 44 12 45 12 46 12 47 12 @.A.B.C.D.E.F.G.
0cf0 48 12 49 12 4a 12 4b 12 4c 12 4d 12 4e 12 4f 12 H.I.J.K.L.M.N.O.
0d00 50 12 51 12 52 12 53 12 54 12 55 12 56 12 57 12 P.Q.R.S.T.U.V.W.
0d10 58 12 59 12 5a 12 5b 12 5c 12 5d 12 5e 12 5f 12 X.Y.Z.[.\.].^._.
0d20 60 12 61 12 62 12 63 12 64 12 65 12 66 12 67 12 `.a.b.c.d.e.f.g.
0d30 68 12 69 12 6a 12 6b 12 6c 12 6d 12 6e 12 6f 12 h.i.j.k.l.m.n.o.
0d40 70 12 71 12 72 12 73 12 74 12 75 12 76 12 77 12 p.q.r.s.t.u.v.w.
0d50 78 12 79 12 7a 12 7b 12 7c 12 7d 12 7e 12 7f 12 x.y.z.{.|.}.~...
0d60 80 12 81 12 82 12 83 12 84 12 85 12 86 12 87 12 ................
0d70 88 12 89 12 8a 12 8b 12 8c 12 8d 12 8e 12 8f 12 ................
0d80 90 12 91 12 92 12 93 12 94 12 95 12 96 12 97 12 ................
0d90 98 12 99 12 9a 12 9b 12 9c 12 9d 12 9e 12 9f 12 ................
0da0 a0 12 a1 12 a2 12 a3 12 a4 12 a5 12 a6 12 a7 12 ................
0db0 a8 12 a9 12 aa 12 ab 12 ac 12 ad 12 ae 12 af 12 ................
0dc0 b0 12 b1 12 b2 12 b3 12 b4 12 b5 12 b6 12 b7 12 ................
0dd0 b8 12 b9 12 ba 12 bb 12 bc 12 bd 12 be 12 bf 12 ................
0de0 c0 12 c1 12 c2 12 c3 12 c4 12 c5 12 c6 12 c7 12 ................
0df0 c8 12 c9 12 ca 12 cb 12 cc 12 cd 12 ce 12 cf 12 ................
0e00 d0 12 d1 12 d2 12 d3 12 d4 12 d5 12 d6 12 d7 12 ................
0e10 d8 12 d9 12 da 12 db 12 dc 12 dd 12 de 12 df 12 ................
0e20 e0 12 e1 12 e2 12 e3 12 e4 12 e5 12 e6 12 e7 12 ................
0e30 e8 12 e9 12 ea 12 eb 12 ec 12 ed 12 ee 12 ef 12 ................
0e40 f0 12 f1 12 f2 12 f3 12 f4 12 f5 12 f6 12 f7 12 ................
0e50 f8 12 f9 12 fa 12 fb 12 fc 12 fd 12 fe 12 ff 12 ................
0e60 00 13 01 13 02 13 03 13 04 13 05 13 06 13 07 13 ................
0e70 08 13 09 13 0a 13 0b 13 0c 13 0d 13 0e 13 0f 13 ................
0e80 10 13 11 13 12 13 13 13 14 13 15 13 16 13 17 13 ................
0e90 18 13 19 13 1a 13 1b 13 1c 13 1d 13 1e 13 1f 13 ................
0ea0 20 13 21 13 22 13 23 13 24 13 25 13 26 13 27 13 .!.".#.$.%.&.'.
0eb0 28 13 29 13 2a 13 2b 13 2c 13 2d 13 2e 13 2f 13 (.).*.+.,.-.../.
0ec0 30 13 31 13 32 13 33 13 34 13 35 13 36 13 37 13 0.1.2.3.4.5.6.7.
0ed0 38 13 39 13 3a 13 3b 13 3c 13 3d 13 3e 13 3f 13 8.9.:.;.<.=.>.?.
0ee0 40 13 41 13 42 13 43 13 44 13 45 13 46 13 47 13 @.A.B.C.D.E.F.G.
0ef0 48 13 49 13 4a 13 4b 13 4c 13 4d 13 4e 13 4f 13 H.I.J.K.L.M.N.O.
0f00 50 13 51 13 52 13 53 13 54 13 55 13 56 13 57 13 P.Q.R.S.T.U.V.W.
0f10 58 13 59 13 5a 13 5b 13 5c 13 5d 13 5e 13 5f 13 X.Y.Z.[.\.].^._.
0f20 60 13 61 13 62 13 63 13 64 13 65 13 66 13 67 13 `.a.b.c.d.e.f.g.
0f30 68 13 69 13 6a 13 6b 13 6c 13 6d 13 6e 13 6f 13 h.i.j.k.l.m.n.o.
0f40 70 13 71 13 72 13 73 13 74 13 75 13 76 13 77 13 p.q.r.s.t.u.v.w.
0f50 78 13 79 13 7a 13 7b 13 7c 13 7d 13 7e 13 7f 13 x.y.z.{.|.}.~...
0f60 80 13 81 13 82 13 83 13 84 13 85 13 86 13 87 13 ................
0f70 88 13 89 13 8a 13 8b 13 8c 13 8d 13 8e 13 8f 13 ................
0f80 90 13 91 13 92 13 93 13 94 13 95 13 96 13 97 13 ................
0f90 98 13 99 13 9a 13 9b 13 9c 13 9d 13 9e 13 9f 13 ................
0fa0 a0 13 a1 13 a2 13 a3 13 a4 13 a5 13 a6 13 a7 13 ................
0fb0 a8 13 a9 13 aa 13 ab 13 ac 13 ad 13 ae 13 af 13 ................
0fc0 b0 13 b1 13 b2 13 b3 13 b4 13 b5 13 b6 13 b7 13 ................
0fd0 b8 13 b9 13 ba 13 bb 13 bc 13 bd 13 be 13 bf 13 ................
0fe0 c0 13 c1 13 c2 13 c3 13 c4 13 c5 13 c6 13 c7 13 ................
0ff0 c8 13 c9 13 ca 13 cb 13 cc 13 cd 13 ce 13 cf 13 ................
1000 d0 13 d1 13 d2 13 d3 13 d4 13 d5 13 d6 13 d7 13 ................
1010 d8 13 d9 13 da 13 db 13 dc 13 dd 13 de 13 df 13 ................
1020 e0 13 e1 13 e2 13 e3 13 e4 13 e5 13 e6 13 e7 13 ................
1030 e8 13 e9 13 ea 13 eb 13 ec 13 ed 13 ee 13 ef 13 ................
1040 f0 13 f1 13 f2 13 f3 13 f4 13 f5 13 f6 13 f7 13 ................
1050 f8 13 f9 13 fa 13 fb 13 fc 13 fd 13 fe 13 ff 13 ................
1060 00 14 01 14 02 14 03 14 04 14 05 14 06 14 07 14 ................
1070 08 14 09 14 0a 14 0b 14 0c 14 0d 14 0e 14 0f 14 ................
1080 10 14 11 14 12 14 13 14 14 14 15 14 16 14 17 14 ................
1090 18 14 19 14 1a 14 1b 14 1c 14 1d 14 1e 14 1f 14 ................
10a0 20 14 21 14 22 14 23 14 24 14 25 14 26 14 27 14 .!.".#.$.%.&.'.
10b0 28 14 29 14 2a 14 2b 14 2c 14 2d 14 2e 14 2f 14 (.).*.+.,.-.../.
10c0 30 14 31 14 32 14 33 14 34 14 35 14 36 14 37 14 0.1.2.3.4.5.6.7.
10d0 38 14 39 14 3a 14 3b 14 3c 14 3d 14 3e 14 3f 14 8.9.:.;.<.=.>.?.
10e0 40 14 41 14 42 14 43 14 44 14 45 14 46 14 47 14 @.A.B.C.D.E.F.G.
10f0 48 14 49 14 4a 14 4b 14 4c 14 4d 14 4e 14 4f 14 H.I.J.K.L.M.N.O.
1100 50 14 51 14 52 14 53 14 54 14 55 14 56 14 57 14 P.Q.R.S.T.U.V.W.
1110 58 14 59 14 5a 14 5b 14 5c 14 5d 14 5e 14 5f 14 X.Y.Z.[.\.].^._.
1120 60 14 61 14 62 14 63 14 64 14 65 14 66 14 67 14 `.a.b.c.d.e.f.g.
1130 68 14 69 14 6a 14 6b 14 6c 14 6d 14 6e 14 6f 14 h.i.j.k.l.m.n.o.
1140 70 14 71 14 72 14 73 14 74 14 75 14 76 14 77 14 p.q.r.s.t.u.v.w.
1150 78 14 79 14 7a 14 7b 14 7c 14 7d 14 7e 14 7f 14 x.y.z.{.|.}.~...
1160 80 14 81 14 82 14 83 14 84 14 85 14 86 14 87 14 ................
1170 88 14 89 14 8a 14 8b 14 8c 14 8d 14 8e 14 8f 14 ................
1180 90 14 91 14 92 14 93 14 94 14 95 14 96 14 97 14 ................
1190 98 14 99 14 9a 14 9b 14 9c 14 9d 14 9e 14 9f 14 ................
11a0 a0 14 a1 14 a2 14 a3 14 a4 14 a5 14 a6 14 a7 14 ................
11b0 a8 14 a9 14 aa 14 ab 14 ac 14 ad 14 ae 14 af 14 ................
11c0 b0 14 b1 14 b2 14 b3 14 b4 14 b5 14 b6 14 b7 14 ................
11d0 b8 14 b9 14 ba 14 bb 14 bc 14 bd 14 be 14 bf 14 ................
11e0 c0 14 c1 14 c2 14 c3 14 c4 14 c5 14 c6 14 c7 14 ................
11f0 c8 14 c9 14 ca 14 cb 14 cc 14 cd 14 ce 14 cf 14 ................
1200 d0 14 d1 14 d2 14 d3 14 d4 14 d5 14 d6 14 d7 14 ................
1210 d8 14 d9 14 da 14 db 14 dc 14 dd 14 de 14 df 14 ................
1220 e0 14 e1 14 e2 14 e3 14 e4 14 e5 14 e6 14 e7 14 ................
1230 e8 14 e9 14 ea 14 eb 14 ec 14 ed 14 ee 14 ef 14 ................
1240 f0 14 f1 14 f2 14 f3 14 f4 14 f5 14 f6 14 f7 14 ................
1250 f8 14 f9 14 fa 14 fb 14 fc 14 fd 14 fe 14 ff 14 ................
1260 00 15 01 15 02 15 03 15 04 15 05 15 06 15 07 15 ................
1270 08 15 09 15 0a 15 0b 15 0c 15 0d 15 0e 15 0f 15 ................
1280 10 15 11 15 12 15 13 15 14 15 15 15 16 15 17 15 ................
1290 18 15 19 15 1a 15 1b 15 1c 15 1d 15 1e 15 1f 15 ................
12a0 20 15 21 15 22 15 23 15 24 15 25 15 26 15 27 15 .!.".#.$.%.&.'.
12b0 28 15 29 15 2a 15 2b 15 2c 15 2d 15 2e 15 2f 15 (.).*.+.,.-.../.
12c0 30 15 31 15 32 15 33 15 34 15 35 15 36 15 37 15 0.1.2.3.4.5.6.7.
12d0 38 15 39 15 3a 15 3b 15 3c 15 3d 15 3e 15 3f 15 8.9.:.;.<.=.>.?.
12e0 40 15 41 15 42 15 43 15 44 15 45 15 46 15 47 15 @.A.B.C.D.E.F.G.
12f0 48 15 49 15 4a 15 4b 15 4c 15 4d 15 4e 15 4f 15 H.I.J.K.L.M.N.O.
1300 50 15 51 15 52 15 53 15 54 15 55 15 56 15 57 15 P.Q.R.S.T.U.V.W.
1310 58 15 59 15 5a 15 5b 15 5c 15 5d 15 5e 15 5f 15 X.Y.Z.[.\.].^._.
1320 60 15 61 15 62 15 63 15 64 15 65 15 66 15 67 15 `.a.b.c.d.e.f.g.
1330 68 15 69 15 6a 15 6b 15 6c 15 6d 15 6e 15 6f 15 h.i.j.k.l.m.n.o.
1340 70 15 71 15 72 15 73 15 74 15 75 15 76 15 77 15 p.q.r.s.t.u.v.w.
1350 78 15 79 15 7a 15 7b 15 7c 15 7d 15 7e 15 7f 15 x.y.z.{.|.}.~...
1360 80 15 81 15 82 15 83 15 84 15 85 15 86 15 87 15 ................
1370 88 15 89 15 8a 15 8b 15 8c 15 8d 15 8e 15 8f 15 ................
1380 90 15 91 15 92 15 93 15 94 15 95 15 96 15 97 15 ................
1390 98 15 99 15 9a 15 9b 15 9c 15 9d 15 9e 15 9f 15 ................
13a0 a0 15 a1 15 a2 15 a3 15 a4 15 a5 15 a6 15 a7 15 ................
13b0 a8 15 a9 15 aa 15 ab 15 ac 15 ad 15 ae 15 af 15 ................
13c0 b0 15 b1 15 b2 15 b3 15 b4 15 b5 15 b6 15 b7 15 ................
13d0 b8 15 b9 15 ba 15 bb 15 bc 15 bd 15 be 15 bf 15 ................
13e0 c0 15 c1 15 c2 15 c3 15 c4 15 c5 15 c6 15 c7 15 ................
13f0 c8 15 c9 15 ca 15 cb 15 cc 15 cd 15 ce 15 cf 15 ................
1400 d0 15 d1 15 d2 15 d3 15 d4 15 d5 15 d6 15 d7 15 ................
1410 d8 15 d9 15 da 15 db 15 dc 15 dd 15 de 15 df 15 ................
1420 e0 15 e1 15 e2 15 e3 15 e4 15 e5 15 e6 15 e7 15 ................
1430 e8 15 e9 15 ea 15 eb 15 ec 15 ed 15 ee 15 ef 15 ................
1440 f0 15 f1 15 f2 15 f3 15 f4 15 f5 15 f6 15 f7 15 ................
1450 f8 15 f9 15 fa 15 fb 15 fc 15 fd 15 fe 15 ff 15 ................
1460 00 16 01 16 02 16 03 16 04 16 05 16 06 16 07 16 ................
1470 08 16 09 16 0a 16 0b 16 0c 16 0d 16 0e 16 0f 16 ................
1480 10 16 11 16 12 16 13 16 14 16 15 16 16 16 17 16 ................
1490 18 16 19 16 1a 16 1b 16 1c 16 1d 16 1e 16 1f 16 ................
14a0 20 16 21 16 22 16 23 16 24 16 25 16 26 16 27 16 .!.".#.$.%.&.'.
14b0 28 16 29 16 2a 16 2b 16 2c 16 2d 16 2e 16 2f 16 (.).*.+.,.-.../.
14c0 30 16 31 16 32 16 33 16 34 16 35 16 36 16 37 16 0.1.2.3.4.5.6.7.
14d0 38 16 39 16 3a 16 3b 16 3c 16 3d 16 3e 16 3f 16 8.9.:.;.<.=.>.?.
14e0 40 16 41 16 42 16 43 16 44 16 45 16 46 16 47 16 @.A.B.C.D.E.F.G.
14f0 48 16 49 16 4a 16 4b 16 4c 16 4d 16 4e 16 4f 16 H.I.J.K.L.M.N.O.
1500 50 16 51 16 52 16 53 16 54 16 55 16 56 16 57 16 P.Q.R.S.T.U.V.W.
1510 58 16 59 16 5a 16 5b 16 5c 16 5d 16 5e 16 5f 16 X.Y.Z.[.\.].^._.
1520 60 16 61 16 62 16 63 16 64 16 65 16 66 16 67 16 `.a.b.c.d.e.f.g.
1530 68 16 69 16 6a 16 6b 16 6c 16 6d 16 6e 16 6f 16 h.i.j.k.l.m.n.o.
1540 70 16 71 16 72 16 73 16 74 16 75 16 76 16 77 16 p.q.r.s.t.u.v.w.
1550 78 16 79 16 7a 16 7b 16 7c 16 7d 16 7e 16 7f 16 x.y.z.{.|.}.~...
1560 80 16 81 16 82 16 83 16 84 16 85 16 86 16 87 16 ................
1570 88 16 89 16 8a 16 8b 16 8c 16 8d 16 8e 16 8f 16 ................
1580 90 16 91 16 92 16 93 16 94 16 95 16 96 16 97 16 ................
1590 98 16 99 16 9a 16 9b 16 9c 16 9d 16 9e 16 9f 16 ................
15a0 a0 16 a1 16 a2 16 a3 16 a4 16 a5 16 a6 16 a7 16 ................
15b0 a8 16 a9 16 aa 16 ab 16 ac 16 ad 16 ae 16 af 16 ................
15c0 b0 16 b1 16 b2 16 b3 16 b4 16 b5 16 b6 16 b7 16 ................
15d0 b8 16 b9 16 ba 16 bb 16 bc 16 bd 16 be 16 bf 16 ................
15e0 c0 16 c1 16 c2 16 c3 16 c4 16 c5 16 c6 16 c7 16 ................
15f0 c8 16 c9 16 ca 16 cb 16 cc 16 cd 16 ce 16 cf 16 ................
1600 d0 16 d1 16 d2 16 d3 16 d4 16 d5 16 d6 16 d7 16 ................
1610 d8 16 d9 16 da 16 db 16 dc 16 dd 16 de 16 df 16 ................
1620 e0 16 e1 16 e2 16 e3 16 e4 16 e5 16 e6 16 e7 16 ................
1630 e8 16 e9 16 ea 16 eb 16 ec 16 ed 16 ee 16 ef 16 ................
1640 f0 16 f1 16 f2 16 f3 16 f4 16 f5 16 f6 16 f7 16 ................
1650 f8 16 f9 16 fa 16 fb 16 fc 16 fd 16 fe 16 ff 16 ................
1660 00 17 01 17 02 17 03 17 04 17 05 17 06 17 07 17 ................
1670 08 17 09 17 0a 17 0b 17 0c 17 0d 17 0e 17 0f 17 ................
1680 10 17 11 17 12 17 13 17 14 17 15 17 16 17 17 17 ................
1690 18 17 19 17 1a 17 1b 17 1c 17 1d 17 1e 17 1f 17 ................
16a0 20 17 21 17 22 17 23 17 24 17 25 17 26 17 27 17 .!.".#.$.%.&.'.
16b0 28 17 29 17 2a 17 2b 17 2c 17 2d 17 2e 17 2f 17 (.).*.+.,.-.../.
16c0 30 17 31 17 32 17 33 17 34 17 35 17 36 17 37 17 0.1.2.3.4.5.6.7.
16d0 38 17 39 17 3a 17 3b 17 3c 17 3d 17 3e 17 3f 17 8.9.:.;.<.=.>.?.
16e0 40 17 41 17 42 17 43 17 44 17 45 17 46 17 47 17 @.A.B.C.D.E.F.G.
16f0 48 17 49 17 4a 17 4b 17 4c 17 4d 17 4e 17 4f 17 H.I.J.K.L.M.N.O.
1700 50 17 51 17 52 17 53 17 54 17 55 17 56 17 57 17 P.Q.R.S.T.U.V.W.
1710 58 17 59 17 5a 17 5b 17 5c 17 5d 17 5e 17 5f 17 X.Y.Z.[.\.].^._.
1720 60 17 61 17 62 17 63 17 64 17 65 17 66 17 67 17 `.a.b.c.d.e.f.g.
1730 68 17 69 17 6a 17 6b 17 6c 17 6d 17 6e 17 6f 17 h.i.j.k.l.m.n.o.
1740 70 17 71 17 72 17 73 17 74 17 75 17 76 17 77 17 p.q.r.s.t.u.v.w.
1750 78 17 79 17 7a 17 7b 17 7c 17 7d 17 7e 17 7f 17 x.y.z.{.|.}.~...
1760 80 17 81 17 82 17 83 17 84 17 85 17 86 17 87 17 ................
1770 88 17 89 17 8a 17 8b 17 8c 17 8d 17 8e 17 8f 17 ................
1780 90 17 91 17 92 17 93 17 94 17 95 17 96 17 97 17 ................
1790 98 17 99 17 9a 17 9b 17 9c 17 9d 17 9e 17 9f 17 ................
17a0 a0 17 a1 17 a2 17 a3 17 a4 17 a5 17 a6 17 a7 17 ................
17b0 a8 17 a9 17 aa 17 ab 17 ac 17 ad 17 ae 17 af 17 ................
17c0 b0 17 b1 17 b2 17 b3 17 b4 17 b5 17 b6 17 b7 17 ................
17d0 b8 17 b9 17 ba 17 bb 17 bc 17 bd 17 be 17 bf 17 ................
17e0 c0 17 c1 17 c2 17 c3 17 c4 17 c5 17 c6 17 c7 17 ................
17f0 c8 17 c9 17 ca 17 cb 17 cc 17 cd 17 ce 17 cf 17 ................
1800 d0 17 d1 17 d2 17 d3 17 d4 17 d5 17 d6 17 d7 17 ................
1810 d8 17 d9 17 da 17 db 17 dc 17 dd 17 de 17 df 17 ................
1820 e0 17 e1 17 e2 17 e3 17 e4 17 e5 17 e6 17 e7 17 ................
1830 e8 17 e9 17 ea 17 eb 17 ec 17 ed 17 ee 17 ef 17 ................
1840 f0 17 f1 17 f2 17 f3 17 f4 17 f5 17 f6 17 f7 17 ................
1850 f8 17 f9 17 fa 17 fb 17 fc 17 fd 17 fe 17 ff 17 ................
1860 00 18 01 18 02 18 03 18 04 18 05 18 06 18 07 18 ................
1870 08 18 09 18 0a 18 0b 18 0c 18 0d 18 0e 18 0f 18 ................
1880 10 18 11 18 12 18 13 18 14 18 15 18 16 18 17 18 ................
1890 18 18 19 18 1a 18 1b 18 1c 18 1d 18 1e 18 1f 18 ................
18a0 20 18 21 18 22 18 23 18 24 18 25 18 26 18 27 18 .!.".#.$.%.&.'.
18b0 28 18 29 18 2a 18 2b 18 2c 18 2d 18 2e 18 2f 18 (.).*.+.,.-.../.
18c0 30 18 31 18 32 18 33 18 34 18 35 18 36 18 37 18 0.1.2.3.4.5.6.7.
18d0 38 18 39 18 3a 18 3b 18 3c 18 3d 18 3e 18 3f 18 8.9.:.;.<.=.>.?.
18e0 40 18 41 18 42 18 43 18 44 18 45 18 46 18 47 18 @.A.B.C.D.E.F.G.
18f0 48 18 49 18 4a 18 4b 18 4c 18 4d 18 4e 18 4f 18 H.I.J.K.L.M.N.O.
1900 50 18 51 18 52 18 53 18 54 18 55 18 56 18 57 18 P.Q.R.S.T.U.V.W.
1910 58 18 59 18 5a 18 5b 18 5c 18 5d 18 5e 18 5f 18 X.Y.Z.[.\.].^._.
1920 60 18 61 18 62 18 63 18 64 18 65 18 66 18 67 18 `.a.b.c.d.e.f.g.
1930 68 18 69 18 6a 18 6b 18 6c 18 6d 18 6e 18 6f 18 h.i.j.k.l.m.n.o.
1940 70 18 71 18 72 18 73 18 74 18 75 18 76 18 77 18 p.q.r.s.t.u.v.w.
1950 78 18 79 18 7a 18 7b 18 7c 18 7d 18 7e 18 7f 18 x.y.z.{.|.}.~...
1960 80 18 81 18 82 18 83 18 84 18 85 18 86 18 87 18 ................
1970 88 18 89 18 8a 18 8b 18 8c 18 8d 18 8e 18 8f 18 ................
1980 90 18 91 18 92 18 93 18 94 18 95 18 96 18 97 18 ................
1990 98 18 99 18 9a 18 9b 18 9c 18 9d 18 9e 18 9f 18 ................
19a0 a0 18 a1 18 a2 18 a3 18 a4 18 a5 18 a6 18 a7 18 ................
19b0 a8 18 a9 18 aa 18 ab 18 ac 18 ad 18 ae 18 af 18 ................
19c0 b0 18 b1 18 b2 18 b3 18 b4 18 b5 18 b6 18 b7 18 ................
19d0 b8 18 b9 18 ba 18 bb 18 bc 18 bd 18 be 18 bf 18 ................
19e0 c0 18 c1 18 c2 18 c3 18 c4 18 c5 18 c6 18 c7 18 ................
19f0 c8 18 c9 18 ca 18 cb 18 cc 18 cd 18 ce 18 cf 18 ................
1a00 d0 18 d1 18 d2 18 d3 18 d4 18 d5 18 d6 18 d7 18 ................
1a10 d8 18 d9 18 da 18 db 18 dc 18 dd 18 de 18 df 18 ................
1a20 e0 18 e1 18 e2 18 e3 18 e4 18 e5 18 e6 18 e7 18 ................
1a30 e8 18 e9 18 ea 18 eb 18 ec 18 ed 18 ee 18 ef 18 ................
1a40 f0 18 f1 18 f2 18 f3 18 f4 18 f5 18 f6 18 f7 18 ................
1a50 f8 18 f9 18 fa 18 fb 18 fc 18 fd 18 fe 18 ff 18 ................
1a60 00 19 01 19 02 19 03 19 04 19 05 19 06 19 07 19 ................
1a70 08 19 09 19 0a 19 0b 19 0c 19 0d 19 0e 19 0f 19 ................
1a80 10 19 11 19 12 19 13 19 14 19 15 19 16 19 17 19 ................
1a90 18 19 19 19 1a 19 1b 19 1c 19 1d 19 1e 19 1f 19 ................
1aa0 20 19 21 19 22 19 23 19 24 19 25 19 26 19 27 19 .!.".#.$.%.&.'.
1ab0 28 19 29 19 2a 19 2b 19 2c 19 2d 19 2e 19 2f 19 (.).*.+.,.-.../.
1ac0 30 19 31 19 32 19 33 19 34 19 35 19 36 19 37 19 0.1.2.3.4.5.6.7.
1ad0 38 19 39 19 3a 19 3b 19 3c 19 3d 19 3e 19 3f 19 8.9.:.;.<.=.>.?.
1ae0 40 19 41 19 42 19 43 19 44 19 45 19 46 19 47 19 @.A.B.C.D.E.F.G.
1af0 48 19 49 19 4a 19 4b 19 4c 19 4d 19 4e 19 4f 19 H.I.J.K.L.M.N.O.
1b00 50 19 51 19 52 19 53 19 54 19 55 19 56 19 57 19 P.Q.R.S.T.U.V.W.
1b10 58 19 59 19 5a 19 5b 19 5c 19 5d 19 5e 19 5f 19 X.Y.Z.[.\.].^._.
1b20 60 19 61 19 62 19 63 19 64 19 65 19 66 19 67 19 `.a.b.c.d.e.f.g.
1b30 68 19 69 19 6a 19 6b 19 6c 19 6d 19 6e 19 6f 19 h.i.j.k.l.m.n.o.
1b40 70 19 71 19 72 19 73 19 74 19 75 19 76 19 77 19 p.q.r.s.t.u.v.w.
1b50 78 19 79 19 7a 19 7b 19 7c 19 7d 19 7e 19 7f 19 x.y.z.{.|.}.~...
1b60 80 19 81 19 82 19 83 19 84 19 85 19 86 19 87 19 ................
1b70 88 19 89 19 8a 19 8b 19 8c 19 8d 19 8e 19 8f 19 ................
1b80 90 19 91 19 92 19 93 19 94 19 95 19 96 19 97 19 ................
1b90 98 19 99 19 9a 19 9b 19 9c 19 9d 19 9e 19 9f 19 ................
1ba0 a0 19 a1 19 a2 19 a3 19 a4 19 a5 19 a6 19 a7 19 ................
1bb0 a8 19 a9 19 aa 19 ab 19 ac 19 ad 19 ae 19 af 19 ................
1bc0 b0 19 b1 19 b2 19 b3 19 b4 19 b5 19 b6 19 b7 19 ................
1bd0 b8 19 b9 19 ba 19 bb 19 bc 19 bd 19 be 19 bf 19 ................
1be0 c0 19 c1 19 c2 19 c3 19 c4 19 c5 19 c6 19 c7 19 ................
1bf0 c8 19 c9 19 ca 19 cb 19 cc 19 cd 19 ce 19 cf 19 ................
1c00 d0 19 d1 19 d2 19 d3 19 d4 19 d5 19 d6 19 d7 19 ................
1c10 d8 19 d9 19 da 19 db 19 dc 19 dd 19 de 19 df 19 ................
1c20 e0 19 e1 19 e2 19 e3 19 e4 19 e5 19 e6 19 e7 19 ................
1c30 e8 19 e9 19 ea 19 eb 19 ec 19 ed 19 ee 19 ef 19 ................
1c40 f0 19 f1 19 f2 19 f3 19 f4 19 f5 19 f6 19 f7 19 ................
1c50 f8 19 f9 19 fa 19 fb 19 fc 19 fd 19 fe 19 ff 19 ................
1c60 00 1a 01 1a 02 1a 03 1a 04 1a 05 1a 06 1a 07 1a ................
1c70 08 1a 09 1a 0a 1a 0b 1a 0c 1a 0d 1a 0e 1a 0f 1a ................
1c80 10 1a 11 1a 12 1a 13 1a 14 1a 15 1a 16 1a 17 1a ................
1c90 18 1a 19 1a 1a 1a 1b 1a 1c 1a 1d 1a 1e 1a 1f 1a ................
1ca0 20 1a 21 1a 22 1a 23 1a 24 1a 25 1a 26 1a 27 1a .!.".#.$.%.&.'.
1cb0 28 1a 29 1a 2a 1a 2b 1a 2c 1a 2d 1a 2e 1a 2f 1a (.).*.+.,.-.../.
1cc0 30 1a 31 1a 32 1a 33 1a 34 1a 35 1a 36 1a 37 1a 0.1.2.3.4.5.6.7.
1cd0 38 1a 39 1a 3a 1a 3b 1a 3c 1a 3d 1a 3e 1a 3f 1a 8.9.:.;.<.=.>.?.
1ce0 40 1a 41 1a 42 1a 43 1a 44 1a 45 1a 46 1a 47 1a @.A.B.C.D.E.F.G.
1cf0 48 1a 49 1a 4a 1a 4b 1a 4c 1a 4d 1a 4e 1a 4f 1a H.I.J.K.L.M.N.O.
1d00 50 1a 51 1a 52 1a 53 1a 54 1a 55 1a 56 1a 57 1a P.Q.R.S.T.U.V.W.
1d10 58 1a 59 1a 5a 1a 5b 1a 5c 1a 5d 1a 5e 1a 5f 1a X.Y.Z.[.\.].^._.
1d20 60 1a 61 1a 62 1a 63 1a 64 1a 65 1a 66 1a 67 1a `.a.b.c.d.e.f.g.
1d30 68 1a 69 1a 6a 1a 6b 1a 6c 1a 6d 1a 6e 1a 6f 1a h.i.j.k.l.m.n.o.
1d40 70 1a 71 1a 72 1a 73 1a 74 1a 75 1a 76 1a 77 1a p.q.r.s.t.u.v.w.
1d50 78 1a 79 1a 7a 1a 7b 1a 7c 1a 7d 1a 7e 1a 7f 1a x.y.z.{.|.}.~...
1d60 80 1a 81 1a 82 1a 83 1a 84 1a 85 1a 86 1a 87 1a ................
1d70 88 1a 89 1a 8a 1a 8b 1a 8c 1a 8d 1a 8e 1a 8f 1a ................
1d80 90 1a 91 1a 92 1a 93 1a 94 1a 95 1a 96 1a 97 1a ................
1d90 98 1a 99 1a 9a 1a 9b 1a 9c 1a 9d 1a 9e 1a 9f 1a ................
1da0 a0 1a a1 1a a2 1a a3 1a a4 1a a5 1a a6 1a a7 1a ................
1db0 a8 1a a9 1a aa 1a ab 1a ac 1a ad 1a ae 1a af 1a ................
1dc0 b0 1a b1 1a b2 1a b3 1a b4 1a b5 1a b6 1a b7 1a ................
1dd0 b8 1a b9 1a ba 1a bb 1a bc 1a bd 1a be 1a bf 1a ................
1de0 c0 1a c1 1a c2 1a c3 1a c4 1a c5 1a c6 1a c7 1a ................
1df0 c8 1a c9 1a ca 1a cb 1a cc 1a cd 1a ce 1a cf 1a ................
1e00 d0 1a d1 1a d2 1a d3 1a d4 1a d5 1a d6 1a d7 1a ................
1e10 d8 1a d9 1a da 1a db 1a dc 1a dd 1a de 1a df 1a ................
1e20 e0 1a e1 1a e2 1a e3 1a e4 1a e5 1a e6 1a e7 1a ................
1e30 e8 1a e9 1a ea 1a eb 1a ec 1a ed 1a ee 1a ef 1a ................
1e40 f0 1a f1 1a f2 1a f3 1a f4 1a f5 1a f6 1a f7 1a ................
1e50 f8 1a f9 1a fa 1a fb 1a fc 1a fd 1a fe 1a ff 1a ................
1e60 00 1b 01 1b 02 1b 03 1b 04 1b 05 1b 06 1b 07 1b ................
1e70 08 1b 09 1b 0a 1b 0b 1b 0c 1b 0d 1b 0e 1b 0f 1b ................
1e80 10 1b 11 1b 12 1b 13 1b 14 1b 15 1b 16 1b 17 1b ................
1e90 18 1b 19 1b 1a 1b 1b 1b 1c 1b 1d 1b 1e 1b 1f 1b ................
1ea0 20 1b 21 1b 22 1b 23 1b 24 1b 25 1b 26 1b 27 1b .!.".#.$.%.&.'.
1eb0 28 1b 29 1b 2a 1b 2b 1b 2c 1b 2d 1b 2e 1b 2f 1b (.).*.+.,.-.../.
1ec0 30 1b 31 1b 32 1b 33 1b 34 1b 35 1b 36 1b 37 1b 0.1.2.3.4.5.6.7.
1ed0 38 1b 39 1b 3a 1b 3b 1b 3c 1b 3d 1b 3e 1b 3f 1b 8.9.:.;.<.=.>.?.
1ee0 40 1b 41 1b 42 1b 43 1b 44 1b 45 1b 46 1b 47 1b @.A.B.C.D.E.F.G.
1ef0 48 1b 49 1b 4a 1b 4b 1b 4c 1b 4d 1b 4e 1b 4f 1b H.I.J.K.L.M.N.O.
1f00 50 1b 51 1b 52 1b 53 1b 54 1b 55 1b 56 1b 57 1b P.Q.R.S.T.U.V.W.
1f10 58 1b 59 1b 5a 1b 5b 1b 5c 1b 5d 1b 5e 1b 5f 1b X.Y.Z.[.\.].^._.
1f20 60 1b 61 1b 62 1b 63 1b 64 1b 65 1b 66 1b 67 1b `.a.b.c.d.e.f.g.
1f30 68 1b 69 1b 6a 1b 6b 1b 6c 1b 6d 1b 6e 1b 6f 1b h.i.j.k.l.m.n.o.
1f40 70 1b 71 1b 72 1b 73 1b 74 1b 75 1b 76 1b 77 1b p.q.r.s.t.u.v.w.
1f50 78 1b 79 1b 7a 1b 7b 1b 7c 1b 7d 1b 7e 1b 7f 1b x.y.z.{.|.}.~...
1f60 80 1b 81 1b 82 1b 83 1b 84 1b 85 1b 86 1b 87 1b ................
1f70 88 1b 89 1b 8a 1b 8b 1b 8c 1b 8d 1b 8e 1b 8f 1b ................
1f80 90 1b 91 1b 92 1b 93 1b 94 1b 95 1b 96 1b 97 1b ................
1f90 98 1b 99 1b 9a 1b 9b 1b 9c 1b 9d 1b 9e 1b 9f 1b ................
1fa0 a0 1b a1 1b a2 1b a3 1b a4 1b a5 1b a6 1b a7 1b ................
1fb0 a8 1b a9 1b aa 1b ab 1b ac 1b ad 1b ae 1b af 1b ................
1fc0 b0 1b b1 1b b2 1b b3 1b b4 1b b5 1b b6 1b b7 1b ................
1fd0 b8 1b b9 1b ba 1b bb 1b bc 1b bd 1b be 1b bf 1b ................
1fe0 c0 1b c1 1b c2 1b c3 1b c4 1b c5 1b c6 1b c7 1b ................
1ff0 c8 1b c9 1b ca 1b cb 1b cc 1b cd 1b ce 1b cf 1b ................
Data: d00bd10bd20bd30bd40bd50bd60bd70bd80bd90bda0bdb0b...
[Length: 8192]
----- Original Message -----
From: Nicholas A. Bellinger <nab@xxxxxxxxxxxxxxx>
To: jrepac@xxxxxxxxx
Cc: "target-devel@xxxxxxxxxxxxxxx" <target-devel@xxxxxxxxxxxxxxx>
Sent: Wednesday, December 21, 2011 2:32 PM
Subject: Re: Test Failure with Header and Data Digest Enabled
On Wed, 2011-12-21 at 08:39 -0800, jrepac@xxxxxxxxx wrote:
> A test failure was noted while testing the target on Windows 2008 x64
> R2 with Iometer. The test setup is as follows:
>
> Emulex iSCSI Initiator
> LIO target setup as a 9GB target with a single portal.
> Disk was partitioned as 8 Windows drives.
>
> Iometer setup - 4 workers / 128 threads per worker / 32 kB transfers
> 50% read.
>
> I ran the test without digest enabled and did not note any failures.
> This was tested over several hours.
>
>
> Both digests were enabled on the initiator and the test was reran.
> Kernel messages immediately started appearing on the screen indicating
> a failure. I checked /var/log/messages and it appears the DataOut CRC
> check failed followed by some kind of recovery failure. First set of
> failures from /var/log/messages are posted below. I plan to simplify
> the test to "writes" only with one worker less threads and investigate
> further.
>
Hi Joe,
Thank you for reporting this issue. I'm going to try to reproduce this
list_del corruption issue by manually triggering an data-out CRC failure
in the same path below.
However, there may also be an issue with the Emulex iSCSI Initiator as
the offset + length below (Offset: 22080, Length: 1384) seems very
strange to me..
Also for reference, I'm not currently aware of any issues with the MSFT
software iSCSI initiator with Header + Data Digest enabled..
Would it be possible for you to re-test with the Windows 2008 x64
software initiator and see if you can reproduce so we can isolate the
Emulex iSCSI offload piece..?
Thanks,
--nab
>
> MODE SENSE: unimplemented page/subpage: 0x1c/0x00
> ITT: 0x0066024e, Offset: 22080, Length: 1384, DataSN: 0x00000002, CRC32C DataDigest 0xb454d12f does not match computed 0xab2e5613
> Unable to recover from DataOUT CRC failure while ERL=0, closing session.
> ------------[ cut here ]------------
> WARNING: at lib/list_debug.c:53 __list_del_entry+0x8d/0x98()
> Hardware name: VMware Virtual Platform
> list_del corruption. prev->next should be ffff880017b42a88, but was ffff880010e00040
> Modules linked in: binfmt_misc bluetooth rfkill tcp_lp iscsi_target_mod target_core_stgt scsi_tgt target_core_pscsi target_core_file target_core_iblock target_core_mod configfs fuse lockd ppdev i2c_piix4 microcode i2c_core parport_pc parport pcspkr vmxnet3 shpchp vmw_balloon sunrpc uinput vmw_pvscsi floppy [last unloaded: scsi_wait_scan]
> Pid: 3162, comm: kworker/0:2 Not tainted 3.2.0-rc4+ #4
> Call Trace:
> [<ffffffff810579c2>] warn_slowpath_common+0x83/0x9b
> [<ffffffffa00f4ace>] ? transport_init_se_cmd+0x104/0x104 [target_core_mod]
> [<ffffffff81057a7d>] warn_slowpath_fmt+0x46/0x48
> [<ffffffff814c1ddc>] ? _raw_spin_unlock_irqrestore+0x17/0x19
> [<ffffffff81237671>] __list_del_entry+0x8d/0x98
> [<ffffffff8123768a>] list_del+0xe/0x2d
> [<ffffffffa00f3aaf>] transport_lun_remove_cmd+0x81/0xa1 [target_core_mod]
> [<ffffffffa00f4dc3>] target_complete_ok_work+0x2f5/0x34e [target_core_mod]
> [<ffffffff8122154c>] ? cfq_init_queue+0x3f8/0x3f8
> [<ffffffffa00f4ace>] ? transport_init_se_cmd+0x104/0x104 [target_core_mod]
> [<ffffffff8106e008>] process_one_work+0x176/0x2a9
> [<ffffffff8106eb16>] worker_thread+0xda/0x15d
> [<ffffffff8106ea3c>] ? manage_workers+0x176/0x176
> [<ffffffff810721ff>] kthread+0x84/0x8c
> [<ffffffff814ca674>] kernel_thread_helper+0x4/0x10
> [<ffffffff8107217b>] ? kthread_worker_fn+0x148/0x148
> [<ffffffff814ca670>] ? gs_change+0x13/0x13
> ---[ end trace ec317e1a71e86303 ]---
> ITT: 0x007a0262, Offset: 30272, Length: 2496, DataSN: 0x00000005, CRC32C DataDigest 0xcb963d4b does not match computed 0x4e2f3e3a
> Unable to recover from DataOUT CRC failure while ERL=0, closing session.
> ------------[ cut here ]------------
> WARNING: at lib/list_debug.c:53 __list_del_entry+0x8d/0x98()
> Hardware name: VMware Virtual Platform
> list_del corruption. prev->next should be ffff880017b43988, but was dead000000100100
> Modules linked in: binfmt_misc bluetooth rfkill tcp_lp iscsi_target_mod target_core_stgt scsi_tgt target_core_pscsi target_core_file target_core_iblock target_core_mod configfs fuse lockd ppdev i2c_piix4 microcode i2c_core parport_pc parport pcspkr vmxnet3 shpchp vmw_balloon sunrpc uinput vmw_pvscsi floppy [last unloaded: scsi_wait_scan]
> Pid: 3162, comm: kworker/0:2 Tainted: G W 3.2.0-rc4+ #4
> Call Trace:
> [<ffffffff810579c2>] warn_slowpath_common+0x83/0x9b
> [<ffffffffa00f4ace>] ? transport_init_se_cmd+0x104/0x104 [target_core_mod]
> [<ffffffff81057a7d>] warn_slowpath_fmt+0x46/0x48
> [<ffffffff814c1ddc>] ? _raw_spin_unlock_irqrestore+0x17/0x19
> [<ffffffff81237671>] __list_del_entry+0x8d/0x98
> [<ffffffff8123768a>] list_del+0xe/0x2d
> [<ffffffffa00f3aaf>] transport_lun_remove_cmd+0x81/0xa1 [target_core_mod]
> [<ffffffffa00f4dc3>] target_complete_ok_work+0x2f5/0x34e [target_core_mod]
> [<ffffffff814c0651>] ? __schedule+0x616/0x644
> [<ffffffffa00f4ace>] ? transport_init_se_cmd+0x104/0x104 [target_core_mod]
> [<ffffffff8106e008>] process_one_work+0x176/0x2a9
> [<ffffffff8106eb16>] worker_thread+0xda/0x15d
> [<ffffffff8106ea3c>] ? manage_workers+0x176/0x176
> [<ffffffff810721ff>] kthread+0x84/0x8c
> [<ffffffff814ca674>] kernel_thread_helper+0x4/0x10
> [<ffffffff8107217b>] ? kthread_worker_fn+0x148/0x148
> [<ffffffff814ca670>] ? gs_change+0x13/0x13
> ---[ end trace ec317e1a71e86304 ]---
> ------------[ cut here ]------------
> --
> To unsubscribe from this list: send the line "unsubscribe target-devel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
