fsck might be able to fix it enough that debugfs and/or mount works, but it may also eliminate all the data, but since you have a clone and no data it is probably worth a shot. fsck -f -y will either get you some data or not and make you have to clone from the copy if you have something else to try. Did you try the "-c" option on debug fs? On Thu, Aug 4, 2022 at 2:07 AM Ken Bass <daytooner@xxxxxxxxx> wrote: > > > That's pretty much it. Whenever any app attempts to read a block from the missing drive, I get the "Buffer I/O error" message. So, even though my recovery apps can scan the LV, marking blocks on the last drive as missing/unknown/etc., they can't display any recovered data - which I know does exist. Looking at raw data from the apps' scans, I can see directory entries, as well as files. I'm sure the inodes and bitmaps are still there for some of these, I just can't really reverse engineer and follow them through. But isn't that what the apps are supposed to do? > > As for debugfs: pretty much the same issue: in order to use it, I need to open the fs. But that (in debugfs) fails as well. So it can't help much. Unless I'm missing something about debugfs. > > The one thing I haven't tried is to use vgreduce to remove the missing PV; but that will also remove the LV as well, which is why I haven't tried it yet. > > Sorry I haven't replied sooner, but it takes a long time (days) to clone, then scan 16Tb... > > So, please any suggestions are greatly appreciated, as well as needed. > > ken > > (I know: No backup; got burned; it hurts; and I will now always have backups. 'Nuf said.) > > > On Thu, Jul 28, 2022 at 3:12 AM Roger James <roger@xxxxxxxxxxxxxxxxxxxxx> wrote: >> >> The procedure outlined should at least get you back to a state where the lv is consistent but with blank sectors where the data is missing. I would suggest using dd to make a backup partition image. Then you can either work on that or the original to mend the fs. >> >> On 27 July 2022 11:50:07 Roger Heflin <rogerheflin@xxxxxxxxx> wrote: >> >>> I don't believe that is going to work. >>> >>> His issue is that the filesystem is refusing to work because of the >>> missing data. >>> >>> man debugfs >>> >>> It will let you manually look at the metadata and structures of the >>> ext2/3/4 fs. You will likely need to use the "-c" option. >>> >>> It will be very manual and you should probably read up on the fs >>> structure a bit. >>> >>> A data recovery company could get most of the data back, but they >>> charge 5k-10k per TB, so likely close to 100k US$. >>> >>> And the issues will be that 1/3 of the metadata was on the missing >>> disk, and some of the data was on the missing disk. >>> >>> I was able to do debugfs /dev/sda2 (my /boot) and do an ls and list >>> out the files and then do a dump <filename> /tmp/junk.out and copy out >>> that file. >>> >>> So the issue will be writing up a script to do lses and find all of >>> the files and dump all of the files to someplace else. >>> >>> On Wed, Jul 27, 2022 at 2:39 AM Roger James <roger@xxxxxxxxxxxxxxxxxxxxx> wrote: >>>> >>>> >>>> Try https://www.linuxsysadmins.com/recover-a-deleted-physical-volume/?amp >>>> >>>> On 26 July 2022 09:16:32 Ken Bass <daytooner@xxxxxxxxx> wrote: >>>>> >>>>> >>>>> (fwiw: I am new to this list, so please bear with me.) >>>>> >>>>> Background: I have a very large (20TB) logical volume consisting of 3 drives. One of those drives unexpectedloy died (isn't that always the case :-)). The drive that failed happened to be the last PV. So I am assuming that there is still 2/3 of the data still intact and, to some extent, recoverable. Although, apparently the ext4 fs is not recognised. >>>>> >>>>> I activated the LV partially (via -P). But running any utility on that (eg: dumpe2fs, e2fsck, ...) I get many of these in dmesg: >>>>> >>>>> "Buffer I/O error on dev dm-0, logical block xxxxxxx, async page read." The thing is, the xxxxxxx block is on the missing drive/pv. >>>>> >>>>> I have also tried some recovery software, but eventually get these same messages, and the data recovered is not really useful. >>>>> >>>>> Please help! How can I get passed that dmesg error, and move on. 14TB recovered is better than 0. >>>>> >>>>> TIA >>>>> ken >>>>> >>>>> >>>>> _______________________________________________ >>>>> linux-lvm mailing list >>>>> linux-lvm@xxxxxxxxxx >>>>> https://listman.redhat.com/mailman/listinfo/linux-lvm >>>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >>>>> >>>> >>>> _______________________________________________ >>>> linux-lvm mailing list >>>> linux-lvm@xxxxxxxxxx >>>> https://listman.redhat.com/mailman/listinfo/linux-lvm >>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >>> >>> >>> _______________________________________________ >>> linux-lvm mailing list >>> linux-lvm@xxxxxxxxxx >>> https://listman.redhat.com/mailman/listinfo/linux-lvm >>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >> >> >> _______________________________________________ >> linux-lvm mailing list >> linux-lvm@xxxxxxxxxx >> https://listman.redhat.com/mailman/listinfo/linux-lvm >> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > > _______________________________________________ > linux-lvm mailing list > linux-lvm@xxxxxxxxxx > https://listman.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ _______________________________________________ linux-lvm mailing list linux-lvm@xxxxxxxxxx https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
