|
We have setup a Compellent SAN with 2 HBA attached to dual
fabrics. Under load when we import a 60GB database, paths fail very often.
Following is the failed path behavior from syslog. Oct 23 02:01:15 db03 kernel: sd 2:0:1:1: SCSI error: return
code = 0x08000002 Oct 23 02:01:15 db03 kernel: sde: Current: sense key:
Aborted Command Oct 23 02:01:15 db03 kernel: Add.
Sense: Internal target failure Oct 23 02:01:15 db03 kernel: Oct 23 02:01:15 db03 kernel: end_request: I/O error, dev
sde, sector 911585239 Oct 23 02:01:15 db03 kernel: device-mapper: multipath:
Failing path 8:64. Oct 23 02:01:15 db03 multipathd: 8:64: mark as failed Oct 23 02:01:15 db03 multipathd: mpath1: remaining active paths:
1 Oct 23 02:01:15 db03 kernel: sd 1:0:3:1: SCSI error: return
code = 0x08000002 Oct 23 02:01:15 db03 kernel: sdc: Current: sense key:
Aborted Command Oct 23 02:01:15 db03 kernel: Add.
Sense: Internal target failure Oct 23 02:01:15 db03 kernel: Oct 23 02:01:15 db03 kernel: end_request: I/O error, dev
sdc, sector 911585239 Oct 23 02:01:15 db03 kernel: device-mapper: multipath:
Failing path 8:32. Oct 23 02:01:16 db03 multipathd: 8:32: mark as failed Oct 23 02:01:16 db03 multipathd: mpath1: remaining active
paths: 0 Oct 23 02:01:19 db03 multipathd: sde: tur checker reports
path is up Oct 23 02:01:19 db03 multipathd: 8:64: reinstated Oct 23 02:01:19 db03 multipathd: mpath1: remaining active
paths: 1 Oct 23 02:01:20 db03 multipathd: sdc: tur checker reports
path is up Oct 23 02:01:20 db03 multipathd: 8:32: reinstated Oct 23 02:01:20 db03 multipathd: mpath1: remaining active
paths: 2 Oct 23 02:01:21 db03 kernel: sd 2:0:1:1: SCSI error: return
code = 0x08000002 Oct 23 02:01:21 db03 kernel: sde: Current: sense key:
Aborted Command Oct 23 02:01:21 db03 kernel: Add.
Sense: Internal target failure Oct 23 02:01:21 db03 kernel: Multipathd segfault during boot and following is from dmesg
output: multipathd[7165]: segfault at 000000000000000a rip
00002aaaaaf51a3d rsp 00007fff03b50090 error 4 sd 2:0:1:1: SCSI error: return code = 0x08000002 sde: Current: sense key: Aborted Command Add. Sense: Internal target failure end_request: I/O error, dev sde, sector 912637903 device-mapper: multipath: Failing path 8:64. sd 1:0:3:1: SCSI error: return code = 0x08000002 sdc: Current: sense key: Aborted Command Add. Sense: Internal target failure end_request: I/O error, dev sdc, sector 915472343 device-mapper: multipath: Failing path 8:32. sd 2:0:1:1: SCSI error: return code = 0x08000002 sde: Current: sense key: Aborted Command Add. Sense: Internal target failure end_request: I/O error, dev sde, sector 915472343 device-mapper: multipath: Failing path 8:64. sd 2:0:1:1: SCSI error: return code = 0x08000002 sde: Current: sense key: Aborted Command Add. Sense: Internal target failure end_request: I/O error, dev sde, sector 919728103 device-mapper: multipath: Failing path 8:64. sd 1:0:3:1: SCSI error: return code = 0x08000002 sdc: Current: sense key: Aborted Command Add. Sense: Internal target failure We have experienced same failures on both RHEL 5.1 and
CentOS. Following is /etc/multipathd.conf defaults {
user_friendly_names yes
path_grouping_policy multibus } devices { device {
vendor "COMPELNT"
product "Compellent Vol"
path_checker tur
polling_interval 10
no_path_retry queue } } blacklist { devnode
"^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode
"^(hd|xvd)[a-z]*" wwid
"*" } # Make sure our multipath devices are enabled. blacklist_exceptions { wwid
"36000d310000e63000000000000000007" wwid
"36000d310000e6300000000000000000c" } # multipath -ll mpath1 (36000d310000e6300000000000000000c) dm-5
COMPELNT,Compellent Vol [size=500G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=2][active] \_ 1:0:3:1 sdc 8:32 [active][ready] \_ 2:0:1:1 sde 8:64 [active][ready] mpath0 (36000d310000e63000000000000000007) dm-0
COMPELNT,Compellent Vol [size=50G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=2][active] \_ 1:0:3:0 sdb 8:16 [active][ready] \_ 2:0:1:0 sdd 8:48 [active][ready] Please let me know if you need more information. This is my
first experience with SAN configuration and I feel that I have missed something
very obvious, because I was not getting meaningful results for those search
results. Thanks, Prakash. |
-- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
