Hi, On Thursday 25 November 2004 10:30, christophe varoqui wrote: > Le jeudi 25 novembre 2004 à 08:55 +0100, Andreas Huck a écrit : > > Hi, > > > > I struggle to setup multipath devices by dmsetup, multipath-tools > > and multipathd. I followed the description at > > http://christophe.varoqui.free.fr/multipath.html > > and refered to all I could find on the web, but some questions > > remain. Maybe you could give some advices? > > > I'd rather remove this doc as it out of date and generate more trouble > than enlightenment. Don't do that. Its really OK for the idea, quite comprehensive and also gives some examples. I'll give you my thoughts on it when I have some minutes left. > > First, how to get a WWID? I have 4 devices with 2 paths each. > > `scsidev -s` should provide it but doesn't: > > > what is the output of "multipath -v2" ? /~# multipath -v 2 # all paths : # all multipaths : # device maps : It's strange that the tool does not show anything although it works basically as I could see with iostat on the underlying scsi devices: After unplugging one FC-cable the traffic switched over onto the failover paths. The traffic was heavy drbd full-sync and a `dd if=/dev/zero bs=1M...` on top of ext3 on top of LVM2 on top of drbd on top of multipath :-) There is, of course, another storage involved. > > > > > So how can I provide WWIDs for /etc/multipath.conf ? > > > That is only necessary for aliases. > You can safely leave the config file multipath section empty. > > > How to identify the devices if for XXX=[a-h]: > > # /sbin/scsi_id -g -p 0x83 -s /block/sdXXX > > 220000003390011d3 > > > And without "-p 0x83" ? fsrv1:# for i in b c d f g h; do echo sd$i: `/sbin/scsi_id -g -p 0x83 -s /block/sd$i`; done sdb: 2200000033900128b sdc: 2200000033900128b sdd: 2200000033900128b sdf: 2200000033900128b sdg: 2200000033900128b sdh: 2200000033900128b fsrv1:# for i in b c d f g h; do echo sd$i: `/sbin/scsi_id -g -s /block/sd$i`; done sdb: 2200000033900128b sdc: 2200000033900128b sdd: 2200000033900128b sdf: 2200000033900128b sdg: 2200000033900128b sdh: 2200000033900128b fsrv1:# for i in b c d f g h; do echo sd$i: `/sbin/scsi_id -g -p 0x80 -s /block/sd$i`; done sdb: SEUROLOGCFC2502 0001cc7420000080e512cc740000000000000000 sdc: SEUROLOGCFC2502 0002cc7420000080e512cc740000000000000000 sdd: SEUROLOGCFC2502 0003cc7420000080e512cc740000000000000000 sdf: SEUROLOGCFC2502 0001cc7420000080e512cc740000000000000000 sdg: SEUROLOGCFC2502 0002cc7420000080e512cc740000000000000000 sdh: SEUROLOGCFC2502 0003cc7420000080e512cc740000000000000000 YES! Thanks. I replaced it but what does that help? Should UUID==WWID? Should I pipe it through ´cut -n -b 17-´? (LMB: the provided "-u" option of scsi_id does not not work "out of the SLES-box".) > > > I.e. all IDs are equal (but configured right, i.e. > > a=e,b=f,c=g,d=h). > > > > Is it wise to take the unique path as ID? : > > # ls -l /sys/block/sdb/device > > /sys/block/sdb/device -> ../../devices/pci0000:04/0000:04:08.0/host0/0:0:0:1 > > > No : we want a unique LU id, not a unique path id. OK, be on the save side. > > > The hardware is not listed in setup_default_hwtable and I wonder > > whether I can define it by multipath.conf at all since sysfs does > > provide the vendor but NOT the "product": > > > Sane defaults are applied when there is no corresponding entry in the > config file nor in setup_default_hwtable. Notably the failover pgpolicy > is selected, which is what you want. > > > # ls /sys/block/sdb/device/ > > . .. block delete detach_state device_blocked generic model > > online power queue_depth rescan rev scsi_level timeout type vendor > > > > Instead I could use 'model "FC2502 "' - but that's not implemented. > > > This is just terminology : "vendor / product /rev" used by most scsi > people, "vendor / model / rev" chosen by dreaming sysfs developers :) > > model == product, so you can add the configlet if you want. > > > I tried a direct approach by dmsetup, but didn't get a device: > > > > # echo "0 274968576 multipath 10 2 1 round-robin 1 0 \ > > /dev/sdb 1 round-robin 1 0 /dev/sdf" | dmsetup create /dev/dm-0 > > k_set_name: Device /dev/dm-0 not found > > Command failed > > > # echo "0 274968576 multipath 10 2 1 round-robin 1 0 \ > /dev/sdb 1 round-robin 1 0 /dev/sdf" | dmsetup create > just_the_map_name_you_want_not_a_path > > > Maybe I am not too familiar with hotplug and udev. > > I already did add a udev rule: > > # grep dm /etc/udev/udev.rules > > KERNEL="dm-[0-9]*", PROGRAM="/sbin/devmap_name %M %m", NAME="%k", SYMLINK="%c" > > > > but > > > > # /sbin/devmap_name 8 16 > > No devices found > > > major:minor are devmap's one, as listed in "dmsetup ls" ? > And the map must exist for the command to work. > > > Although the devices exist: > > brw-rw---- 1 root disk 8, 0 Jun 30 21:04 /dev/sda > > brw-rw---- 1 root disk 8, 16 Jun 30 21:04 /dev/sdb > > brw-rw---- 1 root disk 8, 32 Jun 30 21:04 /dev/sdc > > brw-rw---- 1 root disk 8, 48 Jun 30 21:04 /dev/sdd > > brw-rw---- 1 root disk 8, 64 Jun 30 21:04 /dev/sde > > brw-rw---- 1 root disk 8, 80 Jun 30 21:04 /dev/sdf > > brw-rw---- 1 root disk 8, 96 Jun 30 21:04 /dev/sdg > > brw-rw---- 1 root disk 8, 112 Jun 30 21:04 /dev/sdh > > > > Do I overlook something important here? > > > certainly :) > > > Last but not least, I use > > kernel-smp-2.6.5-7.111 > > udev-021-36.32 > > multipath-tools-0.2.1-1.5 > > device-mapper-1.00.09-17.5 > > from SLES9 and also played around with multipath-tools-0.3.7 from source. > > > I guess you'll have more feedback from Suse folk then. > I don't really know how the combination in SLES9 behave and what patch > pile they applied to the listed products. > > > So if you have some hints for me that would be great. > > > Hope it is enough to get you a bit further. Yes, thanks a lot! One single problem remains: unplugging something in the aktive path leads to a nice failover. How to deal with the correction of the disturbance? If I replug everything, the Emulex reports bravely: Nov 25 16:46:31 fsrv1 kernel: lpfc 0000:04:08.0: 0:1303:LKe:Link Up Event x3 received Data: x3 x0 x8 x0 But just unplugging one of the other (now still active) path components won't bring up the "restored" path but leads to a complete failure of both paths. Is there any way to reset and reuse the revitalized path? I think its similar to the mail from Jims yesterday. Regards, Andreas Huck > > regards,