Geoff Galitz wrote:
> Are folks in the Centos community succesfully using
> device-mapper-multipath? I am looking to deploy it for error handling
> on our iSCSI setup but there seems to be little traffic about this
> package on the Centos forums, as far as I can tell, and there seems
> to be a number of small issues based on my reading the dm-multipath
> developer lists and related resources.
>
> -geoff
Here's my notes from working with our SAN. Please check thoroughly as
these instructions worked for me but I had to learn this from scratch
and there maybe mistakes. One thing I never figured how to achieve was
to rescan an existing ISCSI device for changes if you resize a partition
on the SAN. I have always had to reboot to get the new partition size to
be seen.
###############################
ISCSI notes.
# yum -y install iscsi-initiator-utils lsscsi device-mapper-multipath
# service iscsi start
Add the ISCSI targets
iscsiadm -m discovery -t sendtargets -p 192.168.100.6
iscsiadm -m discovery -t sendtargets -p 192.168.100.2
iscsiadm -m discovery -t sendtargets -p 192.168.100.8
iscsiadm -m discovery -t sendtargets -p 192.168.100.4
# lsscsi
[0:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda
[1:0:0:0] disk COMPELNT Compellent Vol 0306 -
[2:0:0:0] disk COMPELNT Compellent Vol 0306 -
[3:0:0:0] disk COMPELNT Compellent Vol 0306 -
[4:0:0:0] disk COMPELNT Compellent Vol 0306 -
service multipathd start
chkconfig multipathd on
Configuring SAN volumes without reboot
Rescan for ISCSI devices
# iscsiadm -m session -R
Add your partitions For this example I created 2 partitions
Let the running kernel see the new partitions
# partprobe
fdisk /dev/mapper/mpath2
We need to tell the mapper about the new partitions
# ls -l /dev/mapper/mpath2*
brw-rw---- 1 root disk 253, 7 Jan 30 15:42 /dev/mapper/mpath2
# kpartx -l /dev/mapper/mpath2
# ls -l /dev/mapper/mpath2*
brw-rw---- 1 root disk 253, 7 Jan 30 15:42 /dev/mapper/mpath2
brw-rw---- 1 root disk 253, 8 Jan 30 15:43 /dev/mapper/mpath2p1
brw-rw---- 1 root disk 253, 9 Jan 30 15:43 /dev/mapper/mpath2p2
# mke2fs -j /dev/mapper/mpath2p1
# mke2fs -j /dev/mapper/mpath2p2
# iscsiadm -m node
192.168.100.6:3260,0 iqn.2002-03.com.compellent:5000d310000a630a
192.168.100.2:3260,0 iqn.2002-03.com.compellent:5000d310000a6302
192.168.100.4:3260,0 iqn.2002-03.com.compellent:5000d310000a6304
192.168.100.8:3260,0 iqn.2002-03.com.compellent:5000d310000a630c
[root@test2 ~]# iscsiadm -m session
tcp: [12] 192.168.100.6:3260,0
iqn.2002-03.com.compellent:5000d310000a630a
tcp: [13] 192.168.100.2:3260,0
iqn.2002-03.com.compellent:5000d310000a6302
tcp: [14] 192.168.100.4:3260,0
iqn.2002-03.com.compellent:5000d310000a6304
tcp: [15] 192.168.100.8:3260,0
iqn.2002-03.com.compellent:5000d310000a630c
# iscsiadm -m discovery
192.168.100.2:3260 via sendtargets
192.168.100.6:3260 via sendtargets
192.168.100.4:3260 via sendtargets
192.168.100.8:3260 via sendtargets
To automatically mount a file system during startup
you must have the partition entry in /etc/fstab marked with the
"_netdev"
option. For example this would mount a iscsi disk sdb:
/dev/sdb /mnt/iscsi ext3 _netdev 0 0
NOTES WHEN USING LVM WITH MULTIPATH
http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/en-US/RHEL51
0/DM_Multipath/multipath_logical_volumes.html
When you create an LVM logical volume that uses active/passive multipath
arrays as the underlying physical devices, you should include filters in
the lvm.conf to exclude the disks that underlie the multipath devices.
This is because if the array automatically changes the active path to
the passive path when it receives I/O, multipath will failover and
failback whenever LVM scans the passive path if these devices are not
filtered. For active/passive arrays that require a command to make the
passive path active, LVM prints a warning message when this occurs.
To filter all SCSI devices in the multipath configuration file
(lvm.conf), include the following filter in the devices section of the
file.
filter = [ "r/disk/", "r/sd.*/", "a/.*/" ]
A filter to allow sda but disallow all other sd* drives
filter = [ "a|/dev/sda|","r/disk/", "r/sd.*/", "a/.*/" ]
CREATE LVM on top of a SAN MULTI PATH
vi /etc/lvm/lvm.conf
# preferred_names = [ ]
# DAP
preferred_names = [ "^/dev/mpath/", "^/dev/[hs]d" ]
# filter = [ "a/.*/" ]
# DAP
filter = [ "a|/dev/sda|","r/disk/", "r/sd.*/", "a/.*/" ]
pvcreate /dev/mpath/mpath2p1
# pvdisplay /dev/mpath/mpath2p1
--- Physical volume ---
PV Name /dev/mpath/mpath2p1
VG Name VGSAN00
PV Size 1019.72 MB / not usable 3.72 MB
Allocatable yes (but full)
PE Size (KByte) 4096
Total PE 254
Free PE 0
Allocated PE 254
PV UUID ZjqvDp-mxMh-xbuV-CFql-QzaB-cC4l-Eo7RNl
vgcreate VGSAN00 /dev/mpath/mpath2p1
# vgdisplay VGSAN00
--- Volume group ---
VG Name VGSAN00
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 1016.00 MB
PE Size 4.00 MB
Total PE 254
Alloc PE / Size 0 / 0
Free PE / Size 254 / 1016.00 MB
VG UUID bXv8IW-3eJT-RUXB-MLEo-gnfV-4VJv-j598PE
# lvcreate -l 254 -n data VGSAN00
# lvdisplay /dev/VGSAN00/data
--- Logical volume ---
LV Name /dev/VGSAN00/data
VG Name VGSAN00
LV UUID UorNsT-0NuN-kj86-2YGq-gVvG-aJYu-m04bhj
LV Write Access read/write
LV Status available
# open 0
LV Size 1016.00 MB
Current LE 254
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:9
------------------------------------------------
vi /etc/multipath.conf
defaults {
user_friendly_names yes
}
defaults {
udev_dir /dev
polling_interval 10
selector "round-robin 0"
path_grouping_policy multibus
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout /bin/true
path_checker readsector0
rr_min_io 100
rr_weight priorities
failback immediate
no_path_retry fail
user_friendly_name yes
}
devices {
device {
vendor "COMPELNT"
product "Compellent Vol"
path_grouping_policy multibus
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
path_checker readsector0
path_selector "round-robin 0"
hardware_handler "0"
failback 15
rr_weight priorities
no_path_retry queue
}
}
------------------------------------------------
# iscsiadm -m session -i
iscsiadm version 2.0-742
************************************
Session (sid 12) using module tcp:
************************************
TargetName: iqn.2002-03.com.compellent:5000d310000a630a
Portal Group Tag: 0
Network Portal: 192.168.100.6:3260
iSCSI Connection State: LOGGED IN
Internal iscsid Session State: NO CHANGE
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 65536
MaxXmitDataSegmentLength: 65536
FirstBurstLength: 131072
MaxBurstLength: 262144
ImmediateData: Yes
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 12 State: running
scsi12 Channel 00 Id 0 Lun: 0
Attached scsi disk sda State: running
************************************
Session (sid 13) using module tcp:
************************************
TargetName: iqn.2002-03.com.compellent:5000d310000a6302
Portal Group Tag: 0
Network Portal: 192.168.100.2:3260
iSCSI Connection State: LOGGED IN
Internal iscsid Session State: NO CHANGE
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 65536
MaxXmitDataSegmentLength: 65536
FirstBurstLength: 131072
MaxBurstLength: 262144
ImmediateData: Yes
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 13 State: running
scsi13 Channel 00 Id 0 Lun: 0
************************************
Session (sid 14) using module tcp:
************************************
TargetName: iqn.2002-03.com.compellent:5000d310000a6304
Portal Group Tag: 0
Network Portal: 192.168.100.4:3260
iSCSI Connection State: LOGGED IN
Internal iscsid Session State: NO CHANGE
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 65536
MaxXmitDataSegmentLength: 65536
FirstBurstLength: 131072
MaxBurstLength: 262144
ImmediateData: Yes
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 14 State: running
scsi14 Channel 00 Id 0 Lun: 0
************************************
Session (sid 15) using module tcp:
************************************
TargetName: iqn.2002-03.com.compellent:5000d310000a630c
Portal Group Tag: 0
Network Portal: 192.168.100.8:3260
iSCSI Connection State: LOGGED IN
Internal iscsid Session State: NO CHANGE
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 65536
MaxXmitDataSegmentLength: 65536
FirstBurstLength: 131072
MaxBurstLength: 262144
ImmediateData: Yes
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 15 State: running
scsi15 Channel 00 Id 0 Lun: 0
Attached scsi disk sdb State: running
Quick LVM
pvcreate /dev/mapper/mpath3p1
pvcreate /dev/mapper/mpath4p1
vgcreate vg1 /dev/mapper/mpath3p1
lvcreate -l 2559 -n data vg1
mke2fs -j /dev/mapper/vg1-data
mount /dev/mapper/vg1-data /test
vgextend vg1 /dev/mapper/mpath4p1
lvextend -l +100%FREE /dev/vg1/data
resize2fs /dev/vg1/data
LVm cheatsheat
# pvs
PV VG Fmt Attr PSize PFree
/dev/mpath/mpath3p1 vg1 lvm2 a- 10.00G 0
/dev/mpath/mpath4p1 vg1 lvm2 a- 4.99G 0
/dev/sda2 VolGroup00 lvm2 a- 3.88G 0
# vgs
VG #PV #LV #SN Attr VSize VFree
VolGroup00 1 2 0 wz--n- 3.88G 0
vg1 2 1 0 wz--n- 14.99G 0
# lvs
LV VG Attr LSize Origin Snap% Move Log Copy%
LogVol00 VolGroup00 -wi-ao 3.38G
LogVol01 VolGroup00 -wi-ao 512.00M
data vg1 -wi-ao 14.99G
# pvscan
PV /dev/mpath/mpath3p1 VG vg1 lvm2 [10.00 GB / 0 free]
PV /dev/mpath/mpath4p1 VG vg1 lvm2 [4.99 GB / 0 free]
PV /dev/sda2 VG VolGroup00 lvm2 [3.88 GB / 0 free]
Total: 3 [18.86 GB] / in use: 3 [18.86 GB] / in no VG: 0 [0 ]
# vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg1" using metadata type lvm2
Found volume group "VolGroup00" using metadata type lvm2
# lvscan
ACTIVE '/dev/vg1/data' [14.99 GB] inherit
ACTIVE '/dev/VolGroup00/LogVol00' [3.38 GB] inherit
ACTIVE '/dev/VolGroup00/LogVol01' [512.00 MB] inherit
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
