Re: Recovering from a failed pvmove

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Did your operating system have backups/archives of the old cfgs?

Fedora/Redhat keeps around 20 old copies going back typically several
months and so typically one of those can be used.

My archive dir looks like this for the roovg (note I do not do a lot
of vg work, typically
 ls -l /etc/lvm/archive/fedora*
-rw-------. 1 root root 3159 Oct 16  2021
/etc/lvm/archive/fedora_00031-1021957302.vg
-rw-------. 1 root root 2740 Oct 16  2021
/etc/lvm/archive/fedora_00032-1140767599.vg
-rw-------. 1 root root 2331 Oct 16  2021
/etc/lvm/archive/fedora_00033-104560841.vg
-rw-------. 1 root root 2084 Oct 16  2021
/etc/lvm/archive/fedora_00034-239698665.vg
-rw-------. 1 root root 2330 Oct 16  2021
/etc/lvm/archive/fedora_00035-1994061504.vg
-rw-------. 1 root root 1973 Oct 16  2021
/etc/lvm/archive/fedora_00036-974449793.vg
-rw-------. 1 root root 1997 Oct 16  2021
/etc/lvm/archive/fedora_00037-1503932417.vg
-rw-------. 1 root root 1997 Oct 16  2021
/etc/lvm/archive/fedora_00038-951442204.vg
-rw-------. 1 root root 1997 Oct 16  2021
/etc/lvm/archive/fedora_00039-989943813.vg
-rw-------. 1 root root 1968 Oct 16  2021
/etc/lvm/archive/fedora_00040-815563362.vg
-rw-------. 1 root root 1997 Oct 16  2021
/etc/lvm/archive/fedora_00041-1303737065.vg
-rw-------. 1 root root 1968 Oct 16  2021
/etc/lvm/archive/fedora_00042-714254626.vg
-rw-------. 1 root root 1976 Oct 16  2021
/etc/lvm/archive/fedora_00043-858775161.vg
-rw-------. 1 root root 2229 Oct 16  2021
/etc/lvm/archive/fedora_00044-1360584830.vg
-rw-------. 1 root root 2238 Oct 16  2021
/etc/lvm/archive/fedora_00045-1806472194.vg
-rw-------. 1 root root 2218 Oct 16  2021
/etc/lvm/archive/fedora_00046-519515389.vg
-rw-------. 1 root root 1992 Oct 16  2021
/etc/lvm/archive/fedora_00047-1997891375.vg
-rw-------. 1 root root 1988 Oct 16  2021
/etc/lvm/archive/fedora_00048-1666128451.vg
-rw-------. 1 root root 2396 Oct 18  2021
/etc/lvm/archive/fedora_00049-67607509.vg
-rw-------. 1 root root 2396 Oct 18  2021
/etc/lvm/archive/fedora_00050-1102265641.vg
-rw-------. 1 root root 2404 Oct 18  2021
/etc/lvm/archive/fedora_00051-243833122.vg
-rw-------. 1 root root 1738 Oct 18  2021
/etc/lvm/archive/fedora_00052-1619420890.vg
-rw-------. 1 root root 1744 Oct 18  2021
/etc/lvm/archive/fedora_00053-149551096.vg
-rw-------. 1 root root 1748 Nov 11  2021
/etc/lvm/archive/fedora_00054-1785934566.vg
-rw-------. 1 root root 1748 Nov 11  2021
/etc/lvm/archive/fedora_00055-1992369253.vg
-rw-------. 1 root root 1719 Nov 11  2021
/etc/lvm/archive/fedora_00056-26730175.vg
-rw-------. 1 root root 1748 Nov 11  2021
/etc/lvm/archive/fedora_00057-1989410182.vg
-rw-------. 1 root root 1719 Nov 11  2021
/etc/lvm/archive/fedora_00058-1590942582.vg
-rw-------. 1 root root 1748 Nov 11  2021
/etc/lvm/archive/fedora_00059-140743745.vg

I have only personally had to edit the lvm archive I needed to use  a
few times, and that has always been to remove the MISSING tag from
PV's that are no longer missing and the missing tag blocks the archive
from being used.

Most of the time I am correcting VG's I revert to the archive copy
that was taked before the bad steps were done.

There are options that can be put in /etc/lvm/lvm.conf and
/etc/lvm/lvmlocal.conf that can cause the archive's to get collected
if they are not being automatically collected, I am pretty sure I have
seen a few distributions that do not seem to collect the archives that
do typically give you a restore point to revert to (without editing
the file directly).


On Wed, Jun 29, 2022 at 2:32 AM Roger James <roger@xxxxxxxxxxxxxxxxxxxxx> wrote:
>
> I have now managed to fix the problem. I ran vgcfgbackup, then made a copy of the backup for safety purposes. I than hand edited the original backup to remove the missing pv (pv4), the root lv and the pvmove0 lv. I then than vgcfgrestore. Everything is working.
>
> There must be a better way of doing this. Hand editing cfg files is not safe or sensible. What have I missed?
>
> Roger
>
> On 28 June 2022 07:38:48 Roger James <roger@xxxxxxxxxxxxxxxxxxxxx> wrote:
>>
>> Hi,
>>
>> I am struggling to recover from a failed pvmove. Unfortunately I only have a limited knowledge of lvm. I setup my lvm configuration many years ago.
>>
>> I was trying to move a lv to a SSD using pvmove. Unfortunately my brand new SSD choose that moment to fail (never buy cheap SSDs, lesson learnt!").
>>
>> This is the current status.
>>
>> roger@dragon:~$ sudo pvs
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>>   PV VG Fmt Attr PSize PFree
>>   /dev/sda1 wd lvm2 a-- <465.76g 0
>>   /dev/sdb1 wd lvm2 a-- <465.76g <80.45g
>>   /dev/sdc2 wd lvm2 a-- 778.74g 278.74g
>>   /dev/sdd1 wd lvm2 a-- <465.76g 0
>>   [unknown] wd lvm2 a-m <784.49g 685.66g
>> roger@dragon:~$ sudo lvs
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>>   LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
>>   home wd -wi------- 1.46t
>>   root wd -wI-----p- <108.83g
>>   swap wd -wi------- 8.00g
>>   work wd -wi------- 200.00g
>> roger@dragon:~$ sudo vgs
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>>   VG #PV #LV #SN Attr VSize VFree
>>   wd 5 4 0 wz-pn- 2.89t 1.02t
>>
>> This is a recap of what I have tried so far.
>>
>> roger@dragon:~$ sudo pvmove --abort
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>>   LVM command executed by lvmpolld failed.
>>   For more information see lvmpolld messages in syslog or lvmpolld log file.
>> roger@dragon:~$ sudo vgreduce --removemissing wd
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: Partial LV root needs to be repaired or removed.
>>   WARNING: Partial LV pvmove0 needs to be repaired or removed.
>>   There are still partial LVs in VG wd.
>>   To remove them unconditionally use: vgreduce --removemissing --force.
>>   To remove them unconditionally from mirror LVs use: vgreduce --removemissing --mirrorsonly --force.
>>   WARNING: Proceeding to remove empty missing PVs.
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> roger@dragon:~$ sudo lvchange -an wd/root
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> roger@dragon:~$ sudo vgreduce --removemissing wd
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: Partial LV root needs to be repaired or removed.
>>   WARNING: Partial LV pvmove0 needs to be repaired or removed.
>>   There are still partial LVs in VG wd.
>>   To remove them unconditionally use: vgreduce --removemissing --force.
>>   To remove them unconditionally from mirror LVs use: vgreduce --removemissing --mirrorsonly --force.
>>   WARNING: Proceeding to remove empty missing PVs.
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> roger@dragon:~$ sudo lvremove wd/pvmove0
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>>   WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>>   Can't remove locked logical volume wd/pvmove0.
>>
>> I am quite happy to loose the root lv, I just need the home and work lvs. What am I missing?
>>
>> Help!
>>
>> Roger
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@xxxxxxxxxx
>> https://listman.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@xxxxxxxxxx
> https://listman.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
linux-lvm@xxxxxxxxxx
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/




[Index of Archives]     [Gluster Users]     [Kernel Development]     [Linux Clusters]     [Device Mapper]     [Security]     [Bugtraq]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]

  Powered by Linux