Hello,
i tried again whitch Clariion and multipath.
Kernel 2.6.10-udm1
multipath-tools 0.4.2.6
I have one qla2310 FC Controller zoned to SPA and SPB.
I start the multipathd and get the following table
3600601601b121200540b8c1caf4ad911: 0 20971520 multipath 0 1 emc 2 2
round-robin 0 1 1 8:64 1000 round-robin 0 1 1 8:80 1000
What i just tested is :
mount device /mnt
start a long running copy cp -a /usr/* /mnt &
Simulate a dead path by blocking port to SP on my SAN-Switch.
Now i can see that the LUN has trespassed to the remaining SP by dm_emc
module.
The copy command keeps running.
And now I enable the Port to the SP again while the copy is still running.
What i can see is lot of errors from Buffer I/O follwed by ReiserFS
errors and a Kernel Oops.
From now on the Mountpoint is reported as readonly.
I have placed comments (^####) to the messages file.
Jan 19 18:15:04 lxssantm1 kernel: device-mapper: dm-multipath version
1.0.3 loaded
Jan 19 18:15:05 lxssantm1 kernel: device-mapper: dm-emc version 0.0.3
loaded
Jan 19 18:15:33 lxssantm1 multipathd: --------start up--------
Jan 19 18:15:33 lxssantm1 multipathd: read /etc/multipath.conf
Jan 19 18:15:33 lxssantm1 multipathd: Configuration file open problem
Jan 19 18:15:33 lxssantm1 multipathd: add /sbin/scsi_id to binvec
Jan 19 18:15:33 lxssantm1 multipathd: add /sbin/multipath to binvec
Jan 19 18:15:33 lxssantm1 multipathd: ramfs maxsize is 76455
Jan 19 18:15:33 lxssantm1 multipathd: mount ramfs on /var/cache/multipathd
Jan 19 18:15:33 lxssantm1 multipathd: cp /sbin/scsi_id in ramfs
Jan 19 18:15:33 lxssantm1 multipathd: cp /sbin/multipath in ramfs
Jan 19 18:15:33 lxssantm1 multipathd: bind ramfs on /sbin
Jan 19 18:15:33 lxssantm1 multipathd: bind ramfs on /bin
Jan 19 18:15:33 lxssantm1 multipathd: bind ramfs on /tmp
Jan 19 18:15:33 lxssantm1 multipathd: start DM events thread
Jan 19 18:15:33 lxssantm1 multipathd: initial reconfigure multipath maps
Jan 19 18:15:33 lxssantm1 multipathd: path checkers start up
Jan 19 18:15:33 lxssantm1 multipathd: checking paths
Jan 19 18:15:33 lxssantm1 kernel: device-mapper: dm-emc: long trespass
command will be send
Jan 19 18:15:33 lxssantm1 kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Jan 19 18:15:33 lxssantm1 multipathd: refresh devmaps list
Jan 19 18:15:33 lxssantm1 multipathd: devmap
3600601601b121200540b8c1caf4ad911 :
Jan 19 18:15:33 lxssantm1 multipathd: \_ 0 20971520 multipath
Jan 19 18:15:33 lxssantm1 multipathd: refresh failpaths list
Jan 19 18:15:33 lxssantm1 multipathd: dm-0 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: fd0 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop0 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop1 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop2 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop3 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop4 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop5 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop6 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: loop7 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: md0 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram0 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram10 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram11 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram12 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram13 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram14 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram15 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram1 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram2 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram3 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram4 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram5 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram6 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram7 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram8 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: ram9 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: set readsector0 path checker for sda
Jan 19 18:15:33 lxssantm1 multipathd: path checker startup : 8:0
Jan 19 18:15:33 lxssantm1 multipathd: set readsector0 path checker for sdb
Jan 19 18:15:33 lxssantm1 multipathd: path checker startup : 8:16
Jan 19 18:15:33 lxssantm1 multipathd: set readsector0 path checker for sdc
Jan 19 18:15:33 lxssantm1 multipathd: path checker startup : 8:32
Jan 19 18:15:33 lxssantm1 multipathd: set readsector0 path checker for sdd
Jan 19 18:15:33 lxssantm1 multipathd: path checker startup : 8:48
Jan 19 18:15:33 lxssantm1 multipathd: set emc_clariion path checker for sde
Jan 19 18:15:33 lxssantm1 multipathd: path checker startup : 8:64
Jan 19 18:15:33 lxssantm1 multipathd: set emc_clariion path checker for sdf
Jan 19 18:15:33 lxssantm1 kernel: device-mapper: dm-round-robin version
0.2.1 loaded
Jan 19 18:15:33 lxssantm1 multipathd: path checker startup : 8:80
Jan 19 18:15:33 lxssantm1 multipathd: sr0 blacklisted
Jan 19 18:15:33 lxssantm1 multipathd: start up event loops
Jan 19 18:15:33 lxssantm1 multipathd: event checker startup :
3600601601b121200540b8c1caf4ad911
Jan 19 18:15:33 lxssantm1 multipathd: waiter->event_nr = 0
Jan 19 18:15:38 lxssantm1 multipathd: checking paths
Jan 19 18:15:43 lxssantm1 multipathd: checking paths
Jan 19 18:15:44 lxssantm1 udev[30958]: configured rule in
'/etc/udev/rules.d/50-udev.rules' at line 18 applied, added symlink '%c'
Jan 19 18:15:44 lxssantm1 udev[30958]: configured rule in
'/etc/udev/rules.d/50-udev.rules' at line 18 applied, 'dm-0' becomes '%k'
Jan 19 18:15:44 lxssantm1 udev[30958]: creating device node '/dev/dm-0'
Jan 19 18:15:48 lxssantm1 multipathd: checking paths
##### mount
Jan 19 18:16:03 lxssantm1 last message repeated 3 times
Jan 19 18:16:06 lxssantm1 kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Jan 19 18:16:07 lxssantm1 kernel: ReiserFS: dm-0: found reiserfs format
"3.6" with standard journal
Jan 19 18:16:07 lxssantm1 kernel: ReiserFS: dm-0: using ordered data mode
Jan 19 18:16:07 lxssantm1 kernel: ReiserFS: dm-0: journal params: device
dm-0, size 8192, journal first block 18, max trans len 1024, max batch
900, max commit age 30, max trans age 30
Jan 19 18:16:07 lxssantm1 kernel: ReiserFS: dm-0: checking transaction
log (dm-0)
Jan 19 18:16:07 lxssantm1 kernel: ReiserFS: dm-0: replayed 66
transactions in 0 seconds
Jan 19 18:16:07 lxssantm1 kernel: ReiserFS: dm-0: Using r5 hash to sort
names
Jan 19 18:16:08 lxssantm1 multipathd: checking paths
##### mount ende
Jan 19 18:16:43 lxssantm1 last message repeated 7 times
Jan 19 18:16:48 lxssantm1 sshd[31003]: Accepted keyboard-interactive/pam
for root from ::ffff:184.1.3.149 port 3571 ssh2
Jan 19 18:16:48 lxssantm1 sshd[31003]: subsystem request for sftp
Jan 19 18:16:48 lxssantm1 multipathd: checking paths
##### copy -a /usr/* /mnt
Jan 19 18:17:19 lxssantm1 last message repeated 6 times
Jan 19 18:17:34 lxssantm1 last message repeated 3 times
Jan 19 18:17:53 lxssantm1 multipathd: /sbin/multipath -v 0 -S
3600601601b121200540b8c1caf4ad911
Jan 19 18:17:53 lxssantm1 multipathd: devmap event on
3600601601b121200540b8c1caf4ad911
Jan 19 18:17:53 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x20000
Jan 19 18:17:53 lxssantm1 kernel: end_request: I/O error, dev sdf,
sector 2359992
Jan 19 18:17:53 lxssantm1 kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
#### disabled active path copy running
Jan 19 18:19:02 lxssantm1 multipathd: 8:80 : emc_clariion_checker: query
command indicates error
Jan 19 18:19:02 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:02 lxssantm1 last message repeated 8 times
Jan 19 18:19:02 lxssantm1 kernel: device-mapper: dm-emc: long trespass
command will be send
Jan 19 18:19:02 lxssantm1 kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Jan 19 18:19:02 lxssantm1 kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Jan 19 18:19:02 lxssantm1 multipathd: refresh devmaps list
Jan 19 18:19:02 lxssantm1 multipathd: devmap
3600601601b121200540b8c1caf4ad911 :
Jan 19 18:19:02 lxssantm1 multipathd: \_ 0 20971520 multipath
Jan 19 18:19:02 lxssantm1 multipathd: refresh failpaths list
Jan 19 18:19:02 lxssantm1 multipathd: dm-0 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: fd0 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop0 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop1 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop2 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop3 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop4 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop5 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop6 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: loop7 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: md0 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram0 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram10 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram11 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram12 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram13 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram14 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram15 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram1 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram2 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram3 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram4 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram5 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram6 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram7 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram8 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: ram9 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: path checker already active : 8:0
Jan 19 18:19:02 lxssantm1 multipathd: path checker already active : 8:16
Jan 19 18:19:02 lxssantm1 multipathd: path checker already active : 8:32
Jan 19 18:19:02 lxssantm1 multipathd: path checker already active : 8:48
Jan 19 18:19:02 lxssantm1 multipathd: path checker already active : 8:64
Jan 19 18:19:02 lxssantm1 multipathd: path checker already active : 8:80
Jan 19 18:19:02 lxssantm1 multipathd: sr0 blacklisted
Jan 19 18:19:02 lxssantm1 multipathd: start up event loops
Jan 19 18:19:02 lxssantm1 multipathd: event checker startup :
3600601601b121200540b8c1caf4ad911
Jan 19 18:19:02 lxssantm1 multipathd: waiter->event_nr = 2
Jan 19 18:19:07 lxssantm1 multipathd: checking paths
Jan 19 18:19:07 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:12 lxssantm1 multipathd: checking paths
Jan 19 18:19:12 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:17 lxssantm1 multipathd: checking paths
Jan 19 18:19:17 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:22 lxssantm1 multipathd: checking paths
Jan 19 18:19:22 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:27 lxssantm1 multipathd: checking paths
Jan 19 18:19:27 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:32 lxssantm1 multipathd: checking paths
Jan 19 18:19:32 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:37 lxssantm1 multipathd: checking paths
Jan 19 18:19:37 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:42 lxssantm1 multipathd: checking paths
Jan 19 18:19:42 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:47 lxssantm1 multipathd: checking paths
Jan 19 18:19:47 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
#### enable path again copy running
Jan 19 18:19:52 lxssantm1 multipathd: checking paths
Jan 19 18:19:52 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:19:57 lxssantm1 multipathd: checking paths
Jan 19 18:19:57 lxssantm1 kernel: SCSI error : <2 0 2 0> return code =
0x10000
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1577760
Jan 19 18:20:01 lxssantm1 multipathd: /sbin/multipath -v 0 -S
3600601601b121200540b8c1caf4ad911
Jan 19 18:20:01 lxssantm1 multipathd: devmap event on
3600601601b121200540b8c1caf4ad911
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 23721 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1715896
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1715904
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1606328
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1606336
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1605304
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1605312
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1604280
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1604288
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1594360
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1594368
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1593600
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1593608
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592888
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592896
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592784
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592792
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592736
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592744
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592616
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592624
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592424
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592432
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592296
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592304
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592248
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592256
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592128
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1592136
Jan 19 18:20:01 lxssantm1 kernel: SCSI error : <2 0 1 0> return code =
0x20000
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1590152
Jan 19 18:20:01 lxssantm1 kernel: end_request: I/O error, dev sde,
sector 1590160
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6523
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6524
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6525
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6526
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6527
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6528
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6529
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6530
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6531
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: Buffer I/O error on device dm-0,
logical block 6532
Jan 19 18:20:01 lxssantm1 kernel: lost page write due to I/O error on dm-0
Jan 19 18:20:01 lxssantm1 kernel: REISERFS: abort (device dm-0): Journal
write error in flush_commit_list
Jan 19 18:20:01 lxssantm1 kernel: REISERFS: Aborting journal for
filesystem on dm-0
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 23762 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 23778 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 23785 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 23787 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 23800 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 23805 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 25775 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 25815 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 25843 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 25847 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 25857 0x0 SD]
Jan 19 18:20:01 lxssantm1 kernel: ReiserFS: dm-0: warning: vs-13070:
reiserfs_read_locked_inode: i/o failure occurred trying to find stat
data of [17560 25875 0x0 SD]
Jan 19 18:20:01 lxssantm1 multipathd: refresh devmaps list
Jan 19 18:20:01 lxssantm1 multipathd: devmap
3600601601b121200540b8c1caf4ad911 :
Jan 19 18:20:01 lxssantm1 multipathd: \_ 0 20971520 multipath
Jan 19 18:20:01 lxssantm1 multipathd: refresh failpaths list
Jan 19 18:20:01 lxssantm1 kernel: device-mapper: dm-emc: long trespass
command will be send
Jan 19 18:20:01 lxssantm1 kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Jan 19 18:20:01 lxssantm1 multipathd: dm-0 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: fd0 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: loop0 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: loop1 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: loop2 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: Unable to handle kernel paging request
at virtual address f8f41080
Jan 19 18:20:01 lxssantm1 multipathd: loop3 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: printing eip:
Jan 19 18:20:01 lxssantm1 kernel: f8f47e43
Jan 19 18:20:01 lxssantm1 multipathd: loop4 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: loop5 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: *pde = 1fc86067
Jan 19 18:20:01 lxssantm1 multipathd: loop6 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: Oops: 0000 [#1]
Jan 19 18:20:01 lxssantm1 multipathd: loop7 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: SMP
Jan 19 18:20:01 lxssantm1 multipathd: md0 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: Modules linked in: dm_round_robin
dm_emc dm_multipath joydev st sr_mod ide_cd cdrom nvram edd
speedstep_lib freq_table processor ipv6 sg intel_agp agpgart i2c_piix4
i2c_core evdev tulip dm_mod reiserfs qla2300 qla2xxx scsi_transport_fc
gdth aic7xxx sd_mod scsi_mod
Jan 19 18:20:01 lxssantm1 multipathd: ram0 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: ram10 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: ram11 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: ram12 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: ram13 blacklisted
Jan 19 18:20:01 lxssantm1 multipathd: ram14 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: CPU: 1
Jan 19 18:20:01 lxssantm1 multipathd: ram15 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: EIP: 0060:[<f8f47e43>] Not
tainted VLI
Jan 19 18:20:01 lxssantm1 multipathd: ram1 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: EFLAGS: 00010247 (2.6.10)
Jan 19 18:20:01 lxssantm1 multipathd: ram2 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: EIP is at trigger_event+0x3/0x10
[dm_multipath]
Jan 19 18:20:01 lxssantm1 multipathd: ram3 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: eax: f8f41080 ebx: e9519a4c ecx:
00000000 edx: 00000246
Jan 19 18:20:01 lxssantm1 multipathd: ram4 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: esi: dfe30080 edi: e9519a50 ebp:
e95199c0 esp: dff91f60
Jan 19 18:20:01 lxssantm1 multipathd: ram5 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: ds: 007b es: 007b ss: 0068
Jan 19 18:20:01 lxssantm1 multipathd: ram6 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: Process events/1 (pid: 7,
threadinfo=dff90000 task=dffef560)
Jan 19 18:20:01 lxssantm1 multipathd: ram7 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: Stack: c0128a58 dfe300a0 f8f47e40
00000246 dff90000 dfe3008c ffffffff ffffffff
Jan 19 18:20:01 lxssantm1 multipathd: ram8 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: 00000001 00000000 c0116350
00010000 00000000 c18109c0 c1810060 00000000
Jan 19 18:20:01 lxssantm1 multipathd: ram9 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: 00000000 dffef560 c0116350
00100100 00200200 dffef6c4 00000000 00000000
Jan 19 18:20:01 lxssantm1 multipathd: path checker already active : 8:0
Jan 19 18:20:01 lxssantm1 kernel: Call Trace:
Jan 19 18:20:01 lxssantm1 multipathd: path checker already active : 8:16
Jan 19 18:20:01 lxssantm1 kernel: [<c0128a58>] worker_thread+0x1a8/0x230
Jan 19 18:20:01 lxssantm1 multipathd: path checker already active : 8:32
Jan 19 18:20:01 lxssantm1 kernel: [<f8f47e40>] trigger_event+0x0/0x10
[dm_multipath]
Jan 19 18:20:01 lxssantm1 multipathd: path checker already active : 8:48
Jan 19 18:20:01 lxssantm1 kernel: [<c0116350>]
default_wake_function+0x0/0x10
Jan 19 18:20:01 lxssantm1 multipathd: path checker already active : 8:64
Jan 19 18:20:01 lxssantm1 kernel: [<c0116350>]
default_wake_function+0x0/0x10
Jan 19 18:20:01 lxssantm1 multipathd: path checker already active : 8:80
Jan 19 18:20:01 lxssantm1 kernel: [<c01288b0>] worker_thread+0x0/0x230
Jan 19 18:20:01 lxssantm1 multipathd: sr0 blacklisted
Jan 19 18:20:01 lxssantm1 kernel: [<c012c526>] kthread+0x86/0xb0
Jan 19 18:20:01 lxssantm1 multipathd: start up event loops
Jan 19 18:20:01 lxssantm1 kernel: [<c012c4a0>] kthread+0x0/0xb0
Jan 19 18:20:01 lxssantm1 multipathd: event checker startup :
3600601601b121200540b8c1caf4ad911
Jan 19 18:20:01 lxssantm1 kernel: [<c01019b5>]
kernel_thread_helper+0x5/0x10
Jan 19 18:20:01 lxssantm1 multipathd: waiter->event_nr = 3
Jan 19 18:20:01 lxssantm1 kernel: Code: 89 f0 e8 61 fc ff ff 8b 7e 38 85
ff 75 96 8b 46 28 85 c0 74 96 8b 5e 2c 85 db 75 8f c7 44 24 04 01 00 00
00 eb 85 89 f6 8b 40 08 <8b> 00 e9 76 e8 d4 ff 8d b6 00 00 00 00 56 85
d2 53 89 ce 89 c3
Jan 19 18:20:01 lxssantm1 kernel: <6>device-mapper: dm-emc:
emc_pg_init: sending switch-over command
Jan 19 18:20:02 lxssantm1 multipathd: checking paths
Jan 19 18:20:02 lxssantm1 multipathd: 8:80 : emc_clariion_checker: Path
healthy
Jan 19 18:20:02 lxssantm1 multipathd: /sbin/multipath -v 0 -S 8:80
Jan 19 18:20:02 lxssantm1 multipathd: reconfigure 8:80 multipath
Jan 19 18:20:03 lxssantm1 multipathd: refresh devmaps list
Jan 19 18:20:03 lxssantm1 multipathd: devmap
3600601601b121200540b8c1caf4ad911 :
Jan 19 18:20:03 lxssantm1 multipathd: \_ 0 20971520 multipath
Jan 19 18:20:03 lxssantm1 multipathd: refresh failpaths list
Jan 19 18:20:03 lxssantm1 multipathd: dm-0 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: fd0 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop0 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop1 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop2 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop3 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop4 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop5 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop6 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: loop7 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: md0 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram0 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram10 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram11 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram12 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram13 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram14 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram15 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram1 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram2 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram3 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram4 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram5 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram6 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram7 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram8 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: ram9 blacklisted
Jan 19 18:20:03 lxssantm1 multipathd: path checker already active : 8:0
Jan 19 18:20:03 lxssantm1 multipathd: path checker already active : 8:16
Jan 19 18:20:03 lxssantm1 multipathd: path checker already active : 8:32
Jan 19 18:20:03 lxssantm1 multipathd: path checker already active : 8:48
Jan 19 18:20:03 lxssantm1 multipathd: path checker already active : 8:64
Jan 19 18:20:03 lxssantm1 multipathd: path checker already active : 8:80
Jan 19 18:20:03 lxssantm1 multipathd: sr0 blacklisted
Jan 19 18:20:03 lxssantm1 kernel: device-mapper: dm-emc: long trespass
command will be send
Jan 19 18:20:03 lxssantm1 multipathd: start up event loops
Jan 19 18:20:03 lxssantm1 kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Jan 19 18:20:03 lxssantm1 kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Jan 19 18:20:08 lxssantm1 multipathd: checking paths
Jan 19 18:20:43 lxssantm1 last message repeated 7 times