On Wed, 6 Aug 2008, Pasi Kärkkäinen wrote:
Hello!
Is it possible to resize (grow) dm-multipath devices on the fly nowadays?
I googled and found some discussions about the subject, but the conclusion
seemed to be it's not possible.. that was a while ago, so I was wondering if
this has been fixed/implemented..
Using LVM and adding another new LUN/PV is not an option always.. it's a lot
easier to manage the whole thing if it's possible to resize/grow existing
LUNs on the fly.
Sure it is, but it's not pretty. Don't believe the naysayers.
We seem to have a working method for resizing multipath luns from our
NetApp filers. It works on CentOS 4 & 5, for us, and our NetApps.
(2 HBA, times 2 paths on each makes for four SCSI LUN to resize)
The poorly formatted, unpolished, YMMV, no warranty recipe is:
1. Record the state of the dm tables
dmsetup table resize_test | tee orig current new final
2. Split one of the paths
multipath -l
resize_test (360a98000486e5452576f4336486d4c74) dm-2 NETAPP,LUN
[size=1.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 5:0:0:0 sdb 8:16 [active][undef]
\_ 6:0:0:0 sdc 8:32 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 6:0:1:0 sdd 8:48 [active][undef]
\_ 5:0:1:0 sde 8:64 [active][undef]
# Choose a pair: sdb+sdc or sdd+sde. For this, let's choose sdd+sde
# Update current
vi current
0 2097152 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000
# Change this to:
0 2097152 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:16 4000
8:32 4000
# Note this is what happened:
# Remove the second pair beginning with the second "round-robin". We know
it's sdd+sde based on the major/minor pairs.
# Reduce the number of pairs after "queue_if_no_path" from 2 (0 2 1) to 1
(0 1 1)
multipathd -k
del path sdd
del path sde
# Reload the dm table
dmsetup suspend resize_test current; dmsetup reload resize_test current;
dmsetup resume resize_test
# Verify you now just have one set of paths
multipath -l
resize_test (360a98000486e5452576f4336486d4c74) dm-2 NETAPP,LUN
[size=1.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 5:0:0:0 sdb 8:16 [active][undef]
\_ 6:0:0:0 sdc 8:32 [active][undef]
3. Grow the LUN on the SAN (although, this can happen before steps 1 & 2
as well)
(NetApp specific:)
lun resize /vol/resize_test/lun 10g
4. # Update partition table
blockdev --rereadpt /dev/sdd
blockdev --rereadpt /dev/sde
blockdev --getsize /dev/sdd
blockdev --getsize /dev/sde
# The values for getsize in both commands must be the SAME
# Record this value
# The above gave us the value 20971520
# Update the dm table for new
vi new
0 2097152 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000
# Change this to:
0 20971520 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:48 1000
8:64 1000
# Note this is what happened:
# Update the second parameter (num sectors) from 2097152 to 20971520
(obtained from blockdev --getsize)
# Remove the second pair beginning with the second "round-robin"
# Reduce the number of pairs after "queue_if_no_path" from 2 (0 2 1) to 1
(0 1 1)
5. Switch to the new path
multipathd -k
add path sdd
add path sde
del path sdb
del path sdc
# Note: We are adding the INACTIVE Paths first (sdd+sde) and then removing
# the ACTIVE paths (sdb+sdc). This is VERY important. I tried reversing it
by
# deleting ACTIVE first then adding INACTIVE. Fortunately, multipathd kept
one
# of them, refusing to budge.
dmsetup suspend resize_test; dmsetup reload resize_test new; dmsetup
resume resize_test
# Verify we've switched to the new path:
multipath -l
resize_test (360a98000486e5452576f4336486d4c74) dm-2 NETAPP,LUN
[size=10G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
\_ 6:0:1:0 sdd 8:48 [active][undef]
\_ 5:0:1:0 sde 8:64 [active][undef]
# Note: multipath now shows the new size (10G). If it still shows the old
size, chances are
# that one of the block devices in the pair have not had their partition
tables updated.
# TURN BACK! AND MAKE SURE TO REREAD THE PARTITION TABLES!
6. Update the block devices on the inactive path
blockdev --rereadpt /dev/sdb
blockdev --rereadpt /dev/sdc
blockdev --getsize /dev/sdb
blockdev --getsize /dev/sdc
7. Re-add the final path
# Update the final dm table
vi final
0 2097152 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000
# Change to:
0 20971520 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000
# Note this is what happened:
# Update the second parameter (num sectors) from 2097152 to 20971520
# (obtained from blockdev --getsize)
multipathd -k
add path sdb
add path sdc
dmsetup suspend resize_test; dmsetup reload resize_test final; dmsetup
resume resize_test
multipathd -k
reconfigure
# Note: This reconfigure is *ESSENTIAL* as it forces multipath to remember
the new size. Without this step, failing one of the paths
# can cause multipath to reuse the previous block device size and cause
filesystem problems.
8. Resize the FS from userland
ext2online /dev/mapper/resize_test
(or resize2fs in newer )
Pfew! Done!
I think this should have been shared by us (myself, Gino Ledesma, Michael Hsu) a long time ago.
We probably wanted to make a pretty pretty script to do it for us.
There is a slightly more convoluted recipe if you were bold and foolish
enough to partition the LUN (like we once were).
Cheers,
-sharif
--
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel
