We had a customer complaining about disk IO latency so the SAN team used some Hitachi USP magic to migrate the servers existing SAN disks from tier 2 to tier 1. Three nights later I rebooted the server during a maintenance window and the server hung with a console message
“Couldn’t find all physical volumes for volume group vg2
Volume group “vg2” not found”
I commented out the necessary lines in fstab so I could get the server back on the network and ssh into it.
Once in it appeared as if 4 disks were missing.
Metadata does not appear to be corrupt. It just isn’t there.
Fdisk –l shows 4 disks as not having any partition information.
“ doesn't contain a valid partition table “
Did something happen during the tier 2 to tier 1 migration several nights earlier, and I caused it to surface with the reboot? Or do you think something else caused this.
Has anyone experienced storage migration using an Hitachi USP with LVM? Are there any known gotcha’s?
Thanks
Zach
[root@server001 ~]# pvscan
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
Couldn't find device with uuid 'tRZN01-N7bG-SKAw-D9Fd-RxRF-lS4D-U5QfPt'.
Couldn't find device with uuid 'jRwTAu-N0Zs-KBrC-7d6q-uUOA-EHpx-QJeydD'.
Couldn't find device with uuid 'iu1aeD-XVe3-qSHr-cea3-ATGP-ML76-2isZwS'.
Couldn't find device with uuid 'CgJvgU-cND9-Wtfe-cx3Z-g6sH-0tan-Qa21YC'.
PV unknown device VG vg2 lvm2 [58.25 GB / 0 free]
PV unknown device VG vg2 lvm2 [58.25 GB / 0 free]
PV unknown device VG vg2 lvm2 [58.25 GB / 0 free]
PV /dev/dm-30 VG vg2 lvm2 [58.25 GB / 0 free]
PV /dev/dm-37 VG vg2 lvm2 [58.25 GB / 0 free]
PV /dev/dm-33 VG vg2 lvm2 [58.25 GB / 0 free]
PV /dev/dm-35 VG vg2 lvm2 [58.25 GB / 0 free]
PV /dev/dm-40 VG vg2 lvm2 [58.25 GB / 0 free]
PV /dev/dm-38 VG vg2 lvm2 [58.25 GB / 0 free]
PV unknown device VG vg2 lvm2 [58.25 GB / 0 free]
PV /dev/dm-41 VG vg3 lvm2 [992.00 MB / 0 free]
PV /dev/dm-29 VG vg3 lvm2 [992.00 MB / 0 free]
PV /dev/dm-23 VG vg3 lvm2 [992.00 MB / 0 free]
PV /dev/dm-22 VG vg3 lvm2 [992.00 MB / 0 free]
PV /dev/dm-26 VG vg3 lvm2 [992.00 MB / 0 free]
PV /dev/dm-21 VG vg3 lvm2 [992.00 MB / 0 free]
PV /dev/cciss/c0d0p2 VG vg1 lvm2 [68.22 GB / 35.31 GB free]
Total: 17 [656.53 GB] / in use: 17 [656.53 GB] / in no VG: 0 [0 ]
--
Zachary Buckholz, Senior Unix Systems Administrator
University of Phoenix | IT Operations
4025 S. Riverpoint Parkway │ Phoenix, Arizona 85040
phone: 602.557.6809 | cell: 602.377.8023 | email: zach.buckholz@apollogrp.edu
P Please consider the environment before printing this email.
| This message is private and confidential. If you have received it in error, please notify the sender and remove it from your system. |
_______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
