Hello, I am trying to configure multipath for a client over iSCSI using the stgt package in Red Hat Enterprise Linux 6.1 (tgt-1.0.14-2). Here is a very rough sketch of what I am trying to do here: http://i51.tinypic.com/6jir1v.jpg I have a two node Red Hat Cluster where both nodes share a LUN off our SAN. I then encapsulate the shared LUN and create a volume group with the cluster bit set so CLVMD will track metadata changes between cluster members. I finally carve up a 64GB volume out of the volume group and export it as an iSCSI target on both cluster members (targets.conf below). I log into the iSCSI service on both cluster nodes with the client and enumerate the targets as /dev/sda and /dev/sdb. I then start the multipath service on my client with an active/passive configuration. It detects that /dev/sda and /dev/sdb are paths to the same disk which in turn creates a pseudo disk called /dev/mpatha. I am able to format /dev/mpatha and perform work on it. If I change the multipath configuration to active/active where IO is uniformly sent down each iSCSI target at the same time I end up corrupting /dev/mpatha. Why does this occur when my client communicates with iSCSI targets via stgt and not when my client communicates with SAN storage directly using the same native multipath software (dm-multipath)? Could this problem be a result of IO being split between paths not making it to the disk in order/correctly because stgt is using bufered IO rather than direct IO? Any help would be GREATLY appreciated. Cheers, -- SNIP /etc/tgt/targets.conf -- default-driver iscsi <target iqn.1970-01.com.example:netclient-0.disk> incominguser netclient ed86543cf2a15600c93a7c52c0f61b2bae710230 backing-store /dev/cvg00/netclient-0 write-cache off </target> -- SNIP /etc/tgt/targets.conf -- On the client I have the following Active/Passive multipath configuration: -- SNIP /etc/multipath.conf -- defaults { user_friendly_names yes } blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" devnode "^dcssblk[0-9]*" device { vendor "DGC" product "LUNZ" } device { vendor "IBM" product "S/390.*" } # don't count normal SATA devices as multipaths device { vendor "ATA" } # don't count 3ware devices as multipaths device { vendor "3ware" } device { vendor "AMCC" } # nor highpoint devices device { vendor "HPT" } device { vendor QEMU product QEMU_DVD-ROM } wwid "*" } blacklist_exceptions { wwid "1IET 00010001" } multipaths { multipath { uid 0 alias mpatha gid 0 wwid "1IET 00010001" mode 0600 } } -- SNIP /etc/multipath.conf -- -- Greg Procunier UNIX Administrator III - Enterprise Servers and Storage 1 Robert Speck Parkway, Suite 400, Mississauga, Ontario L4Z 4E7 Office: 416-673-3320 Mobile: 647-895-2977 Email: gprocunier@xxxxxxxxxx
CONFIDENTIALITY WARNING This communication, including any attachments, is for the exclusive use of addressee and may contain proprietary and/or confidential information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies. AVERTISSEMENT RELATIF À LA CONFIDENTIALITÉ Ce message, ainsi que les pièces qui y sont jointes, est destiné à l’usage exclusif de la personne à laquelle il s’adresse et peut contenir de l’information personnelle ou confidentielle. Si le lecteur de ce message n’en est pas le destinataire, nous l’avisons par la présente que toute diffusion, distribution, reproduction ou utilisation de son contenu est strictement interdite. Veuillez avertir sur-le-champ l’expéditeur par retour de courrier électronique et supprimez ce message ainsi que toutes les pièces jointes.
