He followed the software RAID and LVM how-tos, eventually building a 930+GB RAID5 volume md0. He then used md0 as the sole PV to build a "raid_discs" VG. This VG was split into three LVs: home, music, and pics. Each LV was formatted EXT3 and set to mount in /etc/fstab:
/dev/raid_discs/music /raid/music ext3 defaults 0 0
/dev/raid_discs/pictures /raid/pictures ext3 defaults 0 0
/dev/raid_discs/home /raid/home ext3 defaults 0 0
This done, he proceeded to copy 260G of media to the server. Thinking everything we living safely on a RAID5 device now, he proceeded to delete the sources. *sigh*
Everything worked fine until he rebooted the server (cleanly, I'm told). After the machine came up, it wouldn't mount his LVs. The RAID device is there:
[root@Server lvmconf]# cat /proc/mdstat
Personalities : [raid5]
read_ahead 1024 sectors
Event: 1
md0 : active raid5 hdh[4] hdg[3] hdf[2] hde[1] hdc[0]
976793600 blocks level 5, 128k chunk, algorithm 2 [5/5] [UUUUU]
unused devices: <none>
From what I can tell, though, it doesn't look like LVM is seeing the RAID device as a PV anymore:
[root@Server lvmconf]# pvdisplay -vv /dev/md0
pvdisplay -- checking physical volume name "/dev/md0"
pvdisplay -- reading data of physical volume "/dev/md0" from disk
pvdisplay -- no physical volume identifier on "/dev/md0"
Am I reading this correctly?
I tried to do a vgcfgrestore and got this:
[root@Server lvmconf]# vgcfgrestore -n raid_discs -vv /dev/md0
vgcfgrestore -- INFO: using backup file "/etc/lvmconf/raid_discs.conf"
vgcfgrestore -- locking logical volume manager
vgcfgrestore -- restoring volume group "raid_discs" from "/etc/lvmconf/raid_discs.conf"
vgcfgrestore -- checking existence of "/etc/lvmconf/raid_discs.conf"
vgcfgrestore -- reading volume group data for "raid_discs" from "/etc/lvmconf/raid_discs.conf"
vgcfgrestore -- reading physical volume data for "raid_discs" from "/etc/lvmconf/raid_discs.conf"
vgcfgrestore -- reading logical volume data for "raid_discs" from "/etc/lvmconf/raid_discs.conf"
vgcfgrestore -- checking volume group consistency of "raid_discs"
vgcfgrestore -- checking volume group consistency of "raid_discs"
vgcfgrestore -- reading physical volume "/dev/md0"
vgcfgrestore -- ERROR "pv_read(): PV identifier invalid" reading physical volume "/dev/md0"
Is there anything that we can do to save his data? I know that with partition tables, if you know them in advance, you can re-write them and, if you're exact, you won't lose any data. Is it possible to do the same with pvcreate? Could I re-init the PV, then have the VG suddenly be happy? Suggestions?
Would it help matters to boot off a live CD like Helix and try recovery that way? I'm a bit hesitant to do that as I'd rather try to get things working with the tools that created the VG in the first place and I'm not sure how nicely LVM2 would play with LVM. OTOH, since we're dealing with base RHEL3, we could also be hitting a bug that was later patched. (After recovery, I'm getting him to change the OS ASAP.)
Thank you so much. I really hope he hasn't borked his pics of his kids.
pete
_______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
