Can someone just confirm that DM-multipath is supposed to failover both ways? i.e. fiber connect is pulled - fails over to other local FC. When the original FC is replaced and the other is pulled - it fails back. Tell me I'm not crazy. Jim On Mon, 29 Nov 2004, Jimmie wrote: > I changed the debug level to 7 in Makefile and recompiled. Don't see a > daemon.log. Is it supposed to be in /var/log? Either way, I'll post the > failover sequence. > > Multipath startup: > Nov 28 12:27:04 nfstest1 multipathd: --------start up-------- > Nov 28 12:27:04 nfstest1 multipathd: read /etc/multipath.conf > Nov 28 12:27:04 nfstest1 multipathd: ramfs maxsize is 94344 > Nov 28 12:27:04 nfstest1 multipathd: start DM events thread > Nov 28 12:27:04 nfstest1 multipathd: path checkers start up > Nov 28 12:27:04 nfstest1 multipathd: initial reconfigure multipath maps > Nov 28 12:27:04 nfstest1 multipathd: refresh devmaps list > Nov 28 12:27:04 nfstest1 multipathd: refresh failpaths list > Nov 28 12:27:04 nfstest1 multipathd: set readsector0 path checker for sdc > Nov 28 12:27:04 nfstest1 multipathd: path checker startup : 8:32 > Nov 28 12:27:04 nfstest1 multipathd: set readsector0 path checker for sdd > Nov 28 12:27:04 nfstest1 multipathd: path checker startup : 8:48 > Nov 28 12:27:04 nfstest1 multipathd: set readsector0 path checker for sde > Nov 28 12:27:04 nfstest1 multipathd: path checker startup : 8:64 > Nov 28 12:27:04 nfstest1 multipathd: set readsector0 path checker for sdf > Nov 28 12:27:04 nfstest1 multipathd: path checker startup : 8:80 > Nov 28 12:27:04 nfstest1 multipathd: set readsector0 path checker for sdg > Nov 28 12:27:04 nfstest1 multipathd: path checker startup : 8:96 > Nov 28 12:27:04 nfstest1 multipathd: set readsector0 path checker for sdh > Nov 28 12:27:04 nfstest1 multipathd: path checker startup : 8:112 > Nov 28 12:27:04 nfstest1 multipathd: start up event loops > Nov 28 12:27:04 nfstest1 multipathd: event checker startup : big01 > > When I pull out the FC from port 1 of the QLogic card: > Nov 29 10:48:55 nfstest1 kernel: qla2300 0000:03:0b.0: LIP reset occured > (f8cb). > Nov 29 10:48:57 nfstest1 kernel: qla2300 0000:03:0b.0: LOOP DOWN detected. > Nov 29 10:49:57 nfstest1 kernel: SCSI error : <1 0 0 1> return code = 0x20000 > Nov 29 10:49:57 nfstest1 kernel: end_request: I/O error, dev sdc, sector 5424 > Nov 29 10:49:57 nfstest1 kernel: end_request: I/O error, dev sdc, sector 5432 > Nov 29 10:49:57 nfstest1 multipathd: devmap event on big01 > Nov 29 10:49:57 nfstest1 multipathd: big01 : reconfigure multipath map > Nov 29 10:50:25 nfstest1 kernel: SCSI error : <1 0 0 1> return code = 0x10000 > Nov 29 10:50:25 nfstest1 kernel: SCSI error : <1 0 0 1> return code = 0x10000 > Nov 29 10:50:26 nfstest1 kernel: SCSI error : <1 0 0 2> return code = 0x10000 > Nov 29 10:50:26 nfstest1 kernel: SCSI error : <1 0 0 3> return code = 0x10000 > Nov 29 10:50:26 nfstest1 kernel: SCSI error : <1 0 0 1> return code = 0x10000 > Nov 29 10:50:26 nfstest1 kernel: SCSI error : <1 0 0 1> return code = 0x10000 > Nov 29 10:50:26 nfstest1 kernel: SCSI error : <1 0 0 2> return code = 0x10000 > > Multipath detects the failure and remaps: > Nov 29 10:50:26 nfstest1 multipathd: refresh devmaps list > Nov 29 10:50:26 nfstest1 multipathd: refresh failpaths list > Nov 29 10:50:26 nfstest1 multipathd: path checker already active : 8:32 > Nov 29 10:50:26 nfstest1 multipathd: path checker already active : 8:48 > Nov 29 10:50:26 nfstest1 multipathd: path checker already active : 8:64 > Nov 29 10:50:26 nfstest1 multipathd: path checker already active : 8:80 > Nov 29 10:50:26 nfstest1 multipathd: path checker already active : 8:96 > Nov 29 10:50:26 nfstest1 multipathd: path checker already active : 8:112 > Nov 29 10:50:26 nfstest1 multipathd: start up event loops > Nov 29 10:50:26 nfstest1 multipathd: event checker startup : big01 > > After I put port1 back in and then pull out port2 (with a couple of minute > wait in between): > Nov 29 10:53:12 nfstest1 kernel: qla2300 0000:03:0b.1: LIP reset occured > (b5b5). > Nov 29 10:53:13 nfstest1 kernel: qla2300 0000:03:0b.1: LOOP DOWN detected. > Nov 29 10:54:10 nfstest1 kernel: SCSI error : <2 0 0 1> return code = 0x20000 > Nov 29 10:54:10 nfstest1 kernel: end_request: I/O error, dev sdf, sector 7944 > Nov 29 10:54:10 nfstest1 kernel: end_request: I/O error, dev sdf, sector 7952 > Nov 29 10:54:11 nfstest1 multipathd: devmap event on big01 > Nov 29 10:54:11 nfstest1 multipathd: big01 : reconfigure multipath map > Nov 29 10:54:11 nfstest1 kernel: Buffer I/O error on device dm-0, logical > block > 1000 > Nov 29 10:54:11 nfstest1 kernel: lost page write due to I/O error on dm-0 > Nov 29 10:54:11 nfstest1 kernel: Aborting journal on device dm-0. > Nov 29 10:54:12 nfstest1 kernel: ext3_abort called. > Nov 29 10:54:12 nfstest1 kernel: EXT3-fs error (device dm-0): > ext3_journal_start > : Detected aborted journal > Nov 29 10:54:12 nfstest1 kernel: Remounting filesystem read-only > > and then a bunch of: > Nov 29 10:54:45 nfstest1 kernel: Buffer I/O error on device dm-0, logical > block > 56983554 > Nov 29 10:54:45 nfstest1 kernel: lost page write due to I/O error on dm-0 > Nov 29 10:54:45 nfstest1 kernel: Buffer I/O error on device dm-0, logical > block > 56983555 > Nov 29 10:54:45 nfstest1 kernel: lost page write due to I/O error on dm-0 > Nov 29 10:54:45 nfstest1 kernel: Buffer I/O error on device dm-0, logical > block > 56983556 > Nov 29 10:54:45 nfstest1 kernel: lost page write due to I/O error on dm-0 > Nov 29 10:54:45 nfstest1 kernel: Buffer I/O error on device dm-0, logical > block > 56983557 > Nov 29 10:54:45 nfstest1 kernel: lost page write due to I/O error on dm-0 > Nov 29 10:54:45 nfstest1 kernel: Buffer I/O error on device dm-0, logical > block > 56983558 > Nov 29 10:54:45 nfstest1 kernel: lost page write due to I/O error on dm-0 > Nov 29 10:54:45 nfstest1 kernel: Buffer I/O error on device dm-0, logical > block > 56983566 > > So basically i get some scsi related errors. Is this normal? Does multipath > failover only work one way? Any ideas? Please help. > > Jimmie > > > > On Thu, 25 Nov 2004, christophe varoqui wrote: > > > The daemon should log in daemon.log > > You can push the debug level to the max and post the trace. > > > > In the mean time, you can also make sure you didn't apply the patchset > > from Mike Christie which used to be appended a the tail of the -udm > > patchset. These patches broke the event model used by the daemon. > > > > regards, > > cvaroqui > > > > Le mercredi 24 novembre 2004 à 17:10 -0500, Jims a écrit : > > > We have a Dell unit with 2 QLogic 23XX series cards which are providing > > > multipathing to 3 EMC volumes. We're looking to have a failover setup (with > > > /dev/sdc and /dev/sdf) so that if one of FC connects is pulled, multipathd > > > will reroute the path to the other card and also be able to reestablish the > > > connection when the Fiber is put back. > > > > > > dmsetup is able to create the device in /udev (/udev/big01) and we're able to > > > mount it. When I pull an FC cable, the mount does indeed failover, however > > > when we put it back in and pull the other, we get a bunch of scsi errors and > > > the mount gets remounted in read-only mode. How can we remedy this? Any > > > similar experiences and/or suggestions? Thanks. > > > > > > By the way, sda and sdb are the system drives. sdd,sde,sdg,sdh are other FC > > > drives that we're not working with right now. > > > > > > our DMsetup table is as follows: > > > > > > DMsetup table <<start>> > > > 0 1885645370 multipath 2 round-robin 1 0 /dev/sdc round-robin 1 0 /dev/sdf > > > DMsetup table <<end>> > > > > > > here is our multipath.conf: > > > > > > multipath.conf <<start>> > > > defaults { > > > multipath_tool "/sbin/multipath -v 0 -S" > > > udev_dir /udev > > > polling_interval 5 > > > default_selector round-robin > > > default_selector_args 0 > > > default_path_grouping_policy failover > > > default_getuid_callout "/sbin/scsi_id -g -u -s" > > > default_prio_callout "/bin/false" > > > } > > > > > > devnode_blacklist { > > > devnode cciss > > > devnode fd > > > devnode hd > > > devnode md > > > devnode dm > > > devnode sr > > > devnode scd > > > devnode st > > > devnode ram > > > devnode raw > > > devnode loop > > > devnode sda > > > devnode sdb > > > } > > > multipaths { > > > multipath { > > > wwid 501566091000 > > > alias big01 > > > path_grouping_policy failover > > > path_selector round-robin > > > } > > > } > > > devices { > > > device { > > > vendor "SEMC " > > > product "SYMMETRIX " > > > path_grouping_policy failover > > > getuid_callout "/sbin/scsi_id -g -u -s" > > > path_checker readsector0 > > > path_selector round-robin > > > } > > > } > > > multipath.conf <<end>> > > > > > > and finally output of multipath -v2 > > > > > > output <<start>> > > > # > > > # all paths : > > > # > > > SEMC_____SYMMETRIX______501566091000 (1 0 0 1) sdc [ready ] (8:32) [SYMMETRIX > > > ] > > > SEMC_____SYMMETRIX______5015660D1000 (1 0 0 2) sdd [ready ] (8:48) [SYMMETRIX > > > ] > > > SEMC_____SYMMETRIX______501566111000 (1 0 0 3) sde [ready ] (8:64) [SYMMETRIX > > > ] > > > SEMC_____SYMMETRIX______501566091000 (2 0 0 1) sdf [ready ] (8:80) [SYMMETRIX > > > ] > > > SEMC_____SYMMETRIX______5015660D1000 (2 0 0 2) sdg [ready ] (8:96) [SYMMETRIX > > > ] > > > SEMC_____SYMMETRIX______501566111000 (2 0 0 3) sdh [ready ] (8:112) [SYMMETRIX > > > ] > > > # > > > # all multipaths : > > > # > > > SEMC_____SYMMETRIX______501566091000 [SYMMETRIX ] > > > \_(1 0 0 1) sdc [ready ] (8:32) > > > \_(2 0 0 1) sdf [ready ] (8:80) > > > SEMC_____SYMMETRIX______5015660D1000 [SYMMETRIX ] > > > \_(1 0 0 2) sdd [ready ] (8:48) > > > \_(2 0 0 2) sdg [ready ] (8:96) > > > SEMC_____SYMMETRIX______501566111000 [SYMMETRIX ] > > > \_(1 0 0 3) sde [ready ] (8:64) > > > \_(2 0 0 3) sdh [ready ] (8:112) > > > # > > > # device maps : > > > # > > > create:SEMC_____SYMMETRIX______501566091000:0 1885655040 multipath 2 > > > round-robin 1 0 8:80 round-robin 1 0 8:32 > > > create:SEMC_____SYMMETRIX______5015660D1000:0 1885655040 multipath 2 > > > round-robin 1 0 8:96 round-robin 1 0 8:48 > > > create:SEMC_____SYMMETRIX______501566111000:0 1885655040 multipath 2 > > > round-robin 1 0 8:112 round-robin 1 0 8:64 > > > output <<end>> > > > > > > Help please. > > > > > > -- > > > > > > dm-devel@xxxxxxxxxx > > > https://www.redhat.com/mailman/listinfo/dm-devel > > >