hi Mark, Thank you so much for the thorough test and response. I created a test volume with a gpt disk label to stay consistent with how the failed volumes were labelled. The only difference between our experiences, that I can tell, is that the pvcreate command failed with an error similar to 'disk doesn't exist, or is being filtered'. I believe its because of the gpt label. I zeroed out the first 512 bytes of the disk label, and continued with the commands (similar to the commands you used): dd if=/dev/zero of=/dev/sdk bs=512 count=1 pvcreate -u 5wJCEA-IDC1-5GhI-jnEs-EpYF-8Uf3-sqPL4O /dev/sdk vgcfgrestore -f /etc/lvm/backup/jetstor642 jetstor642 vgchange -ay I had no problems, and the volumes were recovered. <much celebration> Its possible that my setup is so non-standard, that most people won't be affected (presumably by the upgrade) as I was. Reasons my setup is non-standard: 1) I used ext3 to format >12TB volumes (ext3 has a 8TB limit). 2) I used parted and gpt disk labels 3) I created PV on a whole disk. Luckily, my request for a test environment was approved (after this experience) so I can attempt to replicate the problem and identify the cause of the disk label corruption. Unfortunately, the environment will certainly arrive too late (I do work at a uni ;P) for me to make a timely contribution. Please let me know if I can provide any more information. And thanks again. Best, Julie On 25-09-2009 10.52 +0100, Mark Round wrote: > I just tried this myself.... > > 1. First create a new PV on a whole disk, VG and LV > # pvcreate /dev/sdc > # vgcreate test /dev/sdc > # lvcreate -L2G -n testlv test > > 2. Format the LV, mount it and copy some data to it (just a random > tarball) > # mke2fs -j /dev/test/testlv > # mount /dev/test/testlv /mnt > # tar -C /mnt -xvzf ~/iscsitarget-0.4.17.tar.gz > # umount /mnt > # e2fsck /dev/test/testlv > e2fsck 1.39 (29-May-2006) > /dev/test/testlv: clean, 87/262144 files, 25554/524288 blocks > > 3. So the LV is OK. Now I'll make sure there's a config backup, then > wipe the PV label... > # vgcfgbackup test > Volume group "test" successfully backed up. > # vgchange -an test > 0 logical volume(s) in volume group "test" now active > [root@europa ~]# pvremove -ff /dev/sdc > Really WIPE LABELS from physical volume "/dev/sdc" of volume group > "test" [y/n]? y > WARNING: Wiping physical volume label from /dev/sdc of volume group > "test" > Labels on physical volume "/dev/sdc" successfully wiped > > 4. Now, I'll try to recreate the PV using the backup data, and see if > the contents are intact. > # pvcreate --uuid="A0LDgs-KMlm-QEBR-sGNW-7Rlf-j3aU-x2JUKY" > --restorefile=/etc/lvm/backup/test /dev/sdc > Couldn't find device with uuid > 'A0LDgs-KMlm-QEBR-sGNW-7Rlf-j3aU-x2JUKY'. > Physical volume "/dev/sdc" successfully created > # vgcfgrestore -f /etc/lvm/backup/test test > Restored volume group test > > 5. Check to see if we can see the previously created LV, and mount it > # lvs > LV VG Attr LSize Origin Snap% Move Log Copy% Convert > testlv test -wi--- 2.00G > # vgchange -ay test > 1 logical volume(s) in volume group "test" now active > # e2fsck /dev/test/testlv > e2fsck 1.39 (29-May-2006) > /dev/test/testlv: clean, 87/262144 files, 25554/524288 blocks > # mount /dev/test/testlv /mnt > # ls /mnt > iscsitarget-0.4.17 lost+found > > So, YMMV but it appears from my experiments that this operation should > be safe, and should recover your volumes. I am concerned though about > the news that the RHEL 5.3->5.4 upgrade may have caused this, as we're > looking at making the same upgrade before not too long. Do you have any > suspicion as to why this may have happened ? Have you filed a bug with > Red Hat ? > > Regards, > > -Mark > > -----Original Message----- > From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] > On Behalf Of Julie Ashworth > Sent: 24 September 2009 11:06 > To: linux-lvm@redhat.com > Subject: missing physical volumes after upgrade to rhel 5.4 > > I apologize for the cross-posting (to rhelv5-list). > The lvm list is a more relevant list for my problem, > and I'm sorry I didn't realize this sooner. > > After an upgrade from rhel5.3 -> rhel5.4 (and reboot) > I can no longer see PVs for 3 fibre-channel storage > devices. > > The operating system still see the disk: > ---------------------- > # multipath -l > mpath2 (2001b4d28000064db) dm-1 JetStor,Volume Set # 00 > [size=12T][features=0][hwhandler=0][rw] > \_ round-robin 0 [prio=0][active] > \_ 11:0:1:0 sdj 8:144 [active][undef] > mpath16 (1ACNCorp_FF01000113200019) dm-2 ACNCorp,R_LogVol-despo > [size=15T][features=0][hwhandler=0][rw] > \_ round-robin 0 [prio=0][active] > \_ 11:0:2:0 sdk 8:160 [active][undef] > mpath7 (32800001b4d00cf5b) dm-0 JetStor,Volume Set 416F > [size=12T][features=0][hwhandler=0][rw] > \_ round-robin 0 [prio=0][active] > \_ 11:0:0:0 sdi 8:128 [active][undef] > ---------------------- > > > There are files in /etc/lvm/backup/ that contain the > original volume information, e.g. > ---------------------- > jetstor642 { > id = "0e53Q3-evHX-I5f9-CWqf-NPcw-IqmC-0fVcTO" > seqno = 2 > status = ["RESIZEABLE", "READ", "WRITE"] > flags = [] > extent_size = 8192 # 4 Megabytes > max_lv = 0 > max_pv = 0 > > physical_volumes { > > pv0 { > id = "5wJCEA-IDC1-5GhI-jnEs-EpYF-8Uf3-sqPL4O" > device = "/dev/dm-7" # Hint only > > status = ["ALLOCATABLE"] > flags = [] > dev_size = 31214845952 # 14.5355 Terabytes > pe_start = 384 > pe_count = 3810405 # 14.5355 Terabytes > } > } > ---------------------- > > The devices were formatted using parted on the entire disk, > i.e. I didn't create a partition. > The partition table is "gpt" (possible label types are > "bsd", "dvh", "gpt", "loop", "mac", "msdos", "pc98" > or "sun".) > > > partition table information for one of the devices is below: > -------------------------- > # parted /dev/sdi > GNU Parted 1.8.1 > Using /dev/sdi > Welcome to GNU Parted! Type 'help' to view a list of commands. > (parted) print > > Model: JetStor Volume Set 416F (scsi) > Disk /dev/sdi: 13.0TB > Sector size (logical/physical): 512B/512B > Partition Table: gpt > > Number Start End Size File system Name Flags > -------------------------- > > output of some commands: > > $ pvdisplay > returns nothing (no error) > > $ lvs -a -o +devices > returns nothing (no error) > > $ pvck -vvvvv /dev/sdb > #lvmcmdline.c:915 Processing: pvck -vvvvv /dev/sdb > #lvmcmdline.c:918 O_DIRECT will be used > #config/config.c:950 Setting global/locking_type to 3 > #locking/locking.c:245 Cluster locking selected. > #locking/cluster_locking.c:83 connect() failed on local socket: > Connection > +refused > #config/config.c:955 locking/fallback_to_local_locking not found in > +config: defaulting to 1 > WARNING: Falling back to local file-based locking. > Volume Groups with the clustered attribute will be inaccessible. > #config/config.c:927 Setting global/locking_dir to /var/lock/lvm > #pvck.c:32 Scanning /dev/sdb > #device/dev-cache.c:260 /dev/sdb: Added to device cache > #device/dev-io.c:439 Opened /dev/sdb RO > #device/dev-io.c:260 /dev/sdb: size is 25395814912 sectors > #device/dev-io.c:134 /dev/sdb: block size is 4096 bytes > #filters/filter.c:124 /dev/sdb: Skipping: Partition table > signature > +found > #device/dev-io.c:485 Closed /dev/sdb > #metadata/metadata.c:2337 Device /dev/sdb not found (or ignored by > filtering). > ------------------------- > > from doing google searches, I found this gem to restore a > PV: > pvcreate --uuid "cqH4SD-VrCw-jMsN-GcwH-omCq-ThpE-dO9KmJ" > --restorefile /etc/lvm/backup/vg_04 /dev/sdd1 > > > however, the man page says to 'use with care'. I don't want > to lose data. Can anybody comment on how safe it would be to > run this? > > Thanks in advance, > Julie Ashworth > > > -- > Julie Ashworth <julie.ashworth@berkeley.edu> > Computational Infrastructure for Research Labs, UC Berkeley > http://cirl.berkeley.edu/ > PGP Key ID: 0x17F013D2 > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ---end quoted text--- -- Julie Ashworth <julie.ashworth@berkeley.edu> Computational Infrastructure for Research Labs, UC Berkeley http://cirl.berkeley.edu/ PGP Key ID: 0x17F013D2 _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
