Dear Users -
I am running a Hitachi AMS2500 with 10GB iSCSI SAN, and I am running RedHat Enterprise 5.7 x86_64bit. I installed Multipath and I am seeing strange issues with the box. Basically what appears to be happening is some of my LUNS seem to go into an enabled/enabled status and I/O just queue's up and then eventually times out. I am not sure why this is happening, it is almost like a LUN goes offline for a bit and then I/O back's up and causes major issues. I noticed in the
/var/log/messages
Feb 23 17:59:55 ash-db-3 multipathd: mpath22: load table [0 2147483648 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 70:0 1000 69:240 1000 round-robin 0 2 1 70:32 1000 70:16 1000]
Feb 23 17:59:55 ash-db-3 multipathd: dm-51: add map (uevent)
Feb 23 17:59:55 ash-db-3 multipathd: dm-51: devmap already registered
Feb 23 18:01:51 ash-db-3 kernel: sd 6:0:0:16: timing out command, waited 7s
Feb 23 18:01:51 ash-db-3 multipathd: sdbs: hds prio: SCSI error
Feb 23 18:01:55 ash-db-3 multipathd: mpath17: load table [0 2147483648 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 68:96 1000 68:80 1000 round-robin 0 2 1 68:48 1000 68:64 1000]
Feb 23 18:02:09 ash-db-3 multipathd: dm-37: add map (uevent)
Feb 23 18:02:09 ash-db-3 multipathd: dm-37: devmap already registered
Also if i just Grep for "timing out" i get:
Feb 23 17:09:21 ash-db-3 kernel: sd 6:0:0:2: timing out command, waited 7s
Feb 23 17:11:33 ash-db-3 kernel: sd 8:0:0:10: timing out command, waited 7s
Feb 23 17:18:46 ash-db-3 kernel: sd 7:0:0:15: timing out command, waited 7s
Feb 23 17:22:44 ash-db-3 kernel: sd 8:0:0:2: timing out command, waited 7s
Feb 23 17:27:54 ash-db-3 kernel: sd 6:0:0:10: timing out command, waited 7s
Feb 23 17:34:42 ash-db-3 kernel: sd 8:0:0:2: timing out command, waited 7s
Feb 23 17:35:12 ash-db-3 kernel: sd 8:0:0:17: timing out command, waited 7s
Feb 23 17:37:01 ash-db-3 kernel: sd 6:0:0:2: timing out command, waited 7s
Feb 23 17:37:17 ash-db-3 kernel: sd 6:0:0:5: timing out command, waited 7s
Feb 23 17:39:15 ash-db-3 kernel: sd 6:0:0:16: timing out command, waited 7s
Feb 23 17:44:11 ash-db-3 kernel: sd 6:0:0:2: timing out command, waited 7s
Feb 23 17:46:36 ash-db-3 kernel: sd 8:0:0:2: timing out command, waited 7s
Feb 23 17:48:30 ash-db-3 kernel: sd 6:0:0:7: timing out command, waited 7s
Feb 23 17:51:42 ash-db-3 kernel: sd 6:0:0:16: timing out command, waited 7s
Feb 23 17:56:53 ash-db-3 kernel: sd 6:0:0:15: timing out command, waited 7s
Feb 23 17:59:08 ash-db-3 kernel: sd 6:0:0:2: timing out command, waited 7s
Feb 23 17:59:36 ash-db-3 kernel: sd 6:0:0:23: timing out command, waited 7s
Feb 23 18:01:51 ash-db-3 kernel: sd 6:0:0:16: timing out command, waited 7s
then when I look at "multipath -ll" i see for instance:
mpath12 (360060e80100b05f00530286700000008) dm-7 HITACHI,DF600F
size=1.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=enabled
| |- 8:0:0:2 sdo 8:224 active ready running
| `- 6:0:0:2 sdm 8:192 active ready running
`-+- policy='round-robin 0' prio=0 status=enabled
|- 7:0:0:2 sdl 8:176 active ready running
`- 5:0:0:2 sdn 8:208 active ready running
mpath5 (360060e80100b05f0053028670000000f) dm-29 HITACHI,DF600F
size=1.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| |- 5:0:0:9 sdaq 66:160 active ready running
| `- 7:0:0:9 sdao 66:128 active ready running
`-+- policy='round-robin 0' prio=0 status=enabled
|- 6:0:0:9 sdan 66:112 active ready running
`- 8:0:0:9 sdap 66:144 active ready running
So I wonder if somehow I am getting bit by the "1 queue_if_no_path", however not sure why some LUNS are alwasy ready/enabled, and then some go enabled/enabled. If anyone can provide assistance I would be forever grateful!!
Thanks Much.
Devin Acosta
-- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
