|
Hi, With latest RHEL 4 DM modules and user
space tools, when the Storage controller is rebooted and if “multipath –ll”
is executed, the command is hung for a while (because of retries) and outputs
the following status: 3600508b400102a990000100000710000 [size=4
GB][features="0"][hwhandler="0"] \_ round-robin 0 [enabled] \_ 0:0:0:3 sdc
8:32 [faulty][active] \_ 1:0:0:3 sdi
8:128 [faulty][active] \_ round-robin 0 [enabled] \_ 0:0:1:3 sdf
8:80 [faulty][active] \_ 1:0:1:3 sdl
8:176 [ready ][failed] IO is not exercised during the above test. I
have tested the same by changing the Qlogic parameters for increasing the
timeout/retry values. The following are the various options used for testing. Each
line below have the Qlogic options which are used for building initrd images. options qla2xxx ql2xretrycount=40 options qla2xxx ql2xretrycount=60 options qla2xxx qlport_down_retry=10
ql2xretrycount=60 There is not much change in the time taken
for executing “multipath –ll” command. Just that the
command is hung for 40-60 sec. If we have longer retry counts and if the
controller takes less time like a minute to reboot, the status should have been
updated as the contoller is back online within that time. But in the
subsequent path verification daemon (multipathd) the path status gets updated
properly. The problem is when IO is running on these
devices, after the controller reboots, “dd” reports - dd: reading `/dev/3600508b400102a990000100000710000’:
Input/output error This continues till the next path
verification is done. Attached is the log file. Why does the status not updated correctly? How
should I be setting the HBA (Qlogic in my case) for retry counts and command
timeouts ? Thanks and regards -Murthy |
Aug 17 20:50:21 lxeagle kernel: SCSI error : <1 0 0 3> return code = 0x20000 Aug 17 20:50:21 lxeagle kernel: end_request: I/O error, dev sdi, sector 8027768 Aug 17 20:50:21 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:128. Aug 17 20:50:21 lxeagle kernel: end_request: I/O error, dev sdi, sector 8027776 Aug 17 20:50:21 lxeagle kernel: SCSI error : <1 0 0 3> return code = 0x20000 Aug 17 20:50:21 lxeagle kernel: end_request: I/O error, dev sdi, sector 8027856 Aug 17 20:50:21 lxeagle kernel: end_request: I/O error, dev sdi, sector 8027864 Aug 17 20:50:21 lxeagle multipathd: 8:32: reinstated Aug 17 20:50:21 lxeagle multipathd: red: switch to path group #1 Aug 17 20:50:21 lxeagle multipathd: devmap event (6) on red Aug 17 20:50:21 lxeagle multipathd: 8:48: tur checker reports path is up Aug 17 20:50:21 lxeagle multipathd: 8:48: reinstated Aug 17 20:50:21 lxeagle multipathd: 3600508b400102a9900001000005f0000: switch to path group #1 Aug 17 20:50:21 lxeagle multipathd: 8:64: tur checker reports path is up Aug 17 20:50:21 lxeagle multipathd: 8:64: reinstated Aug 17 20:50:21 lxeagle multipathd: 3600508b400102a9900001000006b0000: switch to path group #1 Aug 17 20:50:21 lxeagle multipathd: 8:80: tur checker reports path is up Aug 17 20:50:21 lxeagle multipathd: 8:80: reinstated Aug 17 20:50:21 lxeagle multipathd: red: switch to path group #1 Aug 17 20:50:21 lxeagle multipathd: 8:96: tur checker reports path is up Aug 17 20:50:21 lxeagle multipathd: 8:96: reinstated Aug 17 20:50:21 lxeagle multipathd: 3600508b400102a9900001000005f0000: switch to path group #1 Aug 17 20:50:21 lxeagle multipathd: 8:112: tur checker reports path is up Aug 17 20:50:22 lxeagle kernel: SCSI error : <0 0 0 3> return code = 0x20000 Aug 17 20:50:22 lxeagle multipathd: 8:112: reinstated Aug 17 20:50:22 lxeagle kernel: end_request: I/O error, dev sdc, sector 8027768 Aug 17 20:50:22 lxeagle multipathd: 3600508b400102a9900001000006b0000: switch to path group #1 Aug 17 20:50:22 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:32. Aug 17 20:50:22 lxeagle multipathd: 8:128: tur checker reports path is up Aug 17 20:50:22 lxeagle kernel: end_request: I/O error, dev sdc, sector 8027776 Aug 17 20:50:22 lxeagle multipathd: 8:128: reinstated Aug 17 20:50:22 lxeagle multipathd: red: switch to path group #1 Aug 17 20:50:22 lxeagle multipathd: 8:144: tur checker reports path is up Aug 17 20:50:22 lxeagle multipathd: 8:144: reinstated Aug 17 20:50:22 lxeagle multipathd: 3600508b400102a9900001000005f0000: switch to path group #1 Aug 17 20:50:22 lxeagle multipathd: 8:160: tur checker reports path is up Aug 17 20:50:22 lxeagle multipathd: 8:160: reinstated Aug 17 20:50:22 lxeagle multipathd: 3600508b400102a9900001000006b0000: switch to path group #1 Aug 17 20:50:22 lxeagle multipathd: 8:176: tur checker reports path is up Aug 17 20:50:22 lxeagle multipathd: 8:176: reinstated Aug 17 20:50:22 lxeagle multipathd: red: switch to path group #1 Aug 17 20:50:22 lxeagle multipathd: devmap event (9) on 3600508b400102a9900001000005f0000 Aug 17 20:50:22 lxeagle multipathd: devmap event (9) on 3600508b400102a9900001000006b0000 Aug 17 20:50:22 lxeagle multipathd: devmap event (9) on red Aug 17 20:50:22 lxeagle multipathd: devmap event (10) on red Aug 17 20:50:23 lxeagle kernel: SCSI error : <1 0 1 3> return code = 0x20000 Aug 17 20:50:23 lxeagle kernel: end_request: I/O error, dev sdl, sector 8027768 Aug 17 20:50:23 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:176. Aug 17 20:50:23 lxeagle kernel: end_request: I/O error, dev sdl, sector 8027776 Aug 17 20:50:33 lxeagle kernel: SCSI error : <0 0 1 3> return code = 0x20000 Aug 17 20:50:33 lxeagle kernel: end_request: I/O error, dev sdf, sector 8027768 Aug 17 20:50:33 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:80. Aug 17 20:50:33 lxeagle kernel: end_request: I/O error, dev sdf, sector 8027776 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003472 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003473 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003474 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003475 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003476 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003477 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003478 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003479 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003480 Aug 17 20:50:33 lxeagle multipathd: 8:0: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003481 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:0 in map 3600508b400102a9900001000005f0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003482 Aug 17 20:50:33 lxeagle multipathd: devmap event (10) on 3600508b400102a9900001000005f0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003483 Aug 17 20:50:33 lxeagle multipathd: 8:16: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003484 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:16 in map 3600508b400102a9900001000006b0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003485 Aug 17 20:50:33 lxeagle multipathd: devmap event (10) on 3600508b400102a9900001000006b0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003486 Aug 17 20:50:33 lxeagle multipathd: 8:32: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003487 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:32 in map red Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003488 Aug 17 20:50:33 lxeagle multipathd: 8:48: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003489 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:48 in map 3600508b400102a9900001000005f0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003490 Aug 17 20:50:33 lxeagle multipathd: 8:64: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003491 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:64 in map 3600508b400102a9900001000006b0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003492 Aug 17 20:50:33 lxeagle multipathd: 8:80: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003493 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:80 in map red Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003494 Aug 17 20:50:33 lxeagle multipathd: 8:96: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003495 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:96 in map 3600508b400102a9900001000005f0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003496 Aug 17 20:50:33 lxeagle multipathd: 8:112: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003497 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:112 in map 3600508b400102a9900001000006b0000 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003498 Aug 17 20:50:33 lxeagle multipathd: 8:128: tur checker reports path is down Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003499 Aug 17 20:50:33 lxeagle multipathd: checker failed path 8:128 in map red Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003500 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003501 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003502 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003503 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003504 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003505 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003506 Aug 17 20:50:33 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003507 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003508 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003509 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003510 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003511 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003512 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003513 Aug 17 20:50:34 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:0. Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003471 Aug 17 20:50:34 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:16. Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1003471 Aug 17 20:50:34 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:48. Aug 17 20:50:34 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:64. Aug 17 20:50:34 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:96. Aug 17 20:50:34 lxeagle kernel: device-mapper: dm-multipath: Failing path 8:112. Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:34 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:35 lxeagle kernel: Buffer I/O device dm-2, logical block 12 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:35 lxeagle multipathd: 8:144: tur checker reports path is down Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:35 lxeagle multipathd: checker failed path 8:144 in map 3600508b400102a9900001000005f0000 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:35 lxeagle multipathd: 8:160: tur checker reports path is down Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:35 lxeagle multipathd: checker failed path 8:160 in map 3600508b400102a9900001000006b0000 Aug 17 20:50:35 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:35 lxeagle multipathd: 8:176: tur checker reports path is down Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:36 lxeagle multipathd: checker failed path 8:176 in map red Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:36 lxeagle multipathd: devmap event (13) on red Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:36 lxeagle multipathd: devmap event (13) on 3600508b400102a9900001000005f0000 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:36 lxeagle multipathd: devmap event (13) on 3600508b400102a9900001000006b0000 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:36 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm- device dm-2, logical block 12 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:37 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm- device dm-2, logical block 12 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:38 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:39 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm- device dm-2, logical block 12 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:40 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm- device dm-2, logical block 12 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:41 lxeagle multipathd: 8:0: tur checker reports path is up Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:41 lxeagle multipathd: 8:0: reinstated Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:41 lxeagle multipathd: 3600508b400102a9900001000005f0000: switch to path group #1 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:41 lxeagle multipathd: devmap event (14) on 3600508b400102a9900001000005f0000 Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:41 lxeagle multipathd: 8:16: tur checker reports path is up Aug 17 20:50:41 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:42 lxeagle multipathd: 8:16: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:42 lxeagle multipathd: 3600508b400102a9900001000006b0000: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:42 lxeagle multipathd: devmap event (14) on 3600508b400102a9900001000006b0000 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:42 lxeagle multipathd: 8:32: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:42 lxeagle multipathd: 8:32: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:42 lxeagle multipathd: red: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:42 lxeagle multipathd: devmap event (14) on red Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:42 lxeagle multipathd: 8:48: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:42 lxeagle multipathd: 8:48: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:42 lxeagle multipathd: 3600508b400102a9900001000005f0000: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:42 lxeagle multipathd: 8:64: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:42 lxeagle multipathd: 8:64: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:42 lxeagle multipathd: 3600508b400102a9900001000006b0000: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:42 lxeagle multipathd: 8:80: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:42 lxeagle multipathd: 8:80: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:42 lxeagle multipathd: red: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:42 lxeagle multipathd: 8:96: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:42 lxeagle multipathd: 8:96: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:42 lxeagle multipathd: 3600508b400102a9900001000005f0000: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:42 lxeagle multipathd: 8:112: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:42 lxeagle multipathd: 8:112: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:42 lxeagle multipathd: 3600508b400102a9900001000006b0000: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:42 lxeagle multipathd: 8:128: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:42 lxeagle multipathd: 8:128: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:42 lxeagle multipathd: red: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:42 lxeagle multipathd: 8:144: tur checker reports path is up Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:42 lxeagle multipathd: 8:144: reinstated Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:42 lxeagle multipathd: 3600508b400102a9900001000005f0000: switch to path group #1 Aug 17 20:50:42 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:43 lxeagle multipathd: 8:160: tur checker reports path is up Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:43 lxeagle multipathd: 8:160: reinstated Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:43 lxeagle multipathd: 3600508b400102a9900001000006b0000: switch to path group #1 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:43 lxeagle multipathd: 8:176: tur checker reports path is up Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:43 lxeagle multipathd: 8:176: reinstated Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:43 lxeagle multipathd: devmap event (17) on 3600508b400102a9900001000005f0000 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:43 lxeagle multipathd: devmap event (17) on 3600508b400102a9900001000006b0000 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:43 lxeagle multipathd: devmap event (17) on red Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 5 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 6 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 7 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 8 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 9 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 10 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 11 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 12 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm- device dm-2, logical block 12 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 13 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 14 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 15 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:43 lxeagle kernel: Buffer I/O error on device dm-2, logical block 0 Aug 17 20:50:44 lxeagle kernel: Buffer I/O error on device dm-2, logical block 1 Aug 17 20:50:44 lxeagle kernel: Buffer I/O error on device dm-2, logical block 2 Aug 17 20:50:44 lxeagle kernel: Buffer I/O error on device dm-2, logical block 3 Aug 17 20:50:44 lxeagle kernel: Buffer I/O error on device dm-2, logical block 4
