Re: Rebuilding mdadm RAID array after OS drive failure

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, Oct 16, 2022 at 6:22 PM anthony <antmbox@xxxxxxxxxxxxxxx> wrote:
>
> On 16/10/2022 22:38, Steve Kolk wrote:
> > I have a system that was running Ubuntu 18.04 server with a 5 drive
> > RAID 6 array using mdadm. I had a hard drive failure, but instead of
> > the array it was the OS hard drive that completely failed. I also do
> > not have access to the original mdadm.conf file.
> >
> > In an attempt to get my array back up, I booted Ubuntu 22.04 from a
> > USB and tried to assemble and mount the array.  It did not work.
> > Trying some diagnostic steps I found online, it seems to be claiming
> > that there is no md superblock on any of the drives.
>
>  From this site? https://raid.wiki.kernel.org/index.php/Linux_Raid
>
> Try https://raid.wiki.kernel.org/index.php/Linux_Raid#When_Things_Go_Wrogn

I set up 3 of the drives with overlays and tried forcing the assembly,
but I still get the same error. I haven't tried the steps under
"Irreversible mdadm failure recovery", is there where I need to go
next?

ubuntu@ubuntu:~$ echo $OVERLAYS
/dev/mapper/sda1 /dev/mapper/sdb1 /dev/mapper/sdc1
ubuntu@ubuntu:~$ sudo mdadm --assemble --force /dev/md1 $OVERLAYS
mdadm: no recogniseable superblock on /dev/mapper/sda1
mdadm: /dev/mapper/sda1 has no superblock - assembly aborted

> >
> > I've included the commands I ran and their output below. Does anyone
> > have any advice on what next steps I can take? I have no reason to
> > think these drives have failed mechanically (much less all of them). I
> > also tried with Ubuntu 18.04 and got the same results.
> >
> If you've got python 2.7, can you run lsdrv? Hopefully that will give us
> some more clues.

ubuntu@ubuntu:~$ ./dev/lsdrv/lsdrv
PCI [ata_piix] 00:1f.2 IDE interface: Intel Corporation 6 Series/C200
Series Chipset Family Desktop SATA Controller (IDE mode, ports 0-3)
(rev 05)
├scsi 0:0:0:0 ATA      WDC WD100EFAX-68
│└sda 9.10t [8:0] Empty/Unknown
│ ├sda1 9.10t [8:1] Empty/Unknown
│ │└dm-2 9.10t [253:2] Empty/Unknown
│ └sda9 8.00m [8:9] Empty/Unknown
├scsi 0:0:1:0 ATA      WDC WD100EFAX-68
│└sdb 9.10t [8:16] Empty/Unknown
│ ├sdb1 9.10t [8:17] Empty/Unknown
│ │└dm-0 9.10t [253:0] Empty/Unknown
│ └sdb9 8.00m [8:25] Empty/Unknown
├scsi 1:0:0:0 ATA      WDC WD100EFAX-68
│└sdc 9.10t [8:32] Empty/Unknown
│ ├sdc1 9.10t [8:33] Empty/Unknown
│ │└dm-1 9.10t [253:1] Empty/Unknown
│ └sdc9 8.00m [8:41] Empty/Unknown
└scsi 1:0:1:0 ATA      WDC WD100EFAX-68
 └sdd 9.10t [8:48] Empty/Unknown
  ├sdd1 9.10t [8:49] Empty/Unknown
  └sdd9 8.00m [8:57] Empty/Unknown
PCI [ata_piix] 00:1f.5 IDE interface: Intel Corporation 6 Series/C200
Series Chipset Family Desktop SATA Controller (IDE mode, ports 4-5)
(rev 05)
└scsi 2:0:0:0 ATA      WDC WD100EFAX-68
 └sde 9.10t [8:64] Empty/Unknown
  ├sde1 9.10t [8:65] Empty/Unknown
  └sde9 8.00m [8:73] Empty/Unknown
PCI [pata_via] 03:00.0 IDE interface: VIA Technologies, Inc. VT6415
PATA IDE Host Controller
└scsi 4:x:x:x [Empty]
USB [usb-storage] Bus 002 Device 003: ID 0930:6545 Toshiba Corp.
Kingston DataTraveler 102/2.0 / HEMA Flash Drive 2 GB / PNY Attache
4GB Stick {000AEBFFB4C15B8A130501A6}
└scsi 6:0:0:0 Kingston DataTraveler 2.0
 └sdf 7.57g [8:80] Empty/Unknown
  └sdf1 7.57g [8:81] Empty/Unknown
   └Mounted as /dev/sdf1 @ /cdrom
Other Block Devices
├loop0 2.13g [7:0] Empty/Unknown
│└Mounted as /dev/loop0 @ /rofs
├loop1 61.96m [7:1] Empty/Unknown
│└Mounted as /dev/loop1 @ /snap/core20/1587
├loop2 4.00k [7:2] Empty/Unknown
│└Mounted as /dev/loop2 @ /snap/bare/5
├loop3 163.29m [7:3] Empty/Unknown
│└Mounted as /dev/loop3 @ /snap/firefox/1635
├loop4 91.69m [7:4] Empty/Unknown
│└Mounted as /dev/loop4 @ /snap/gtk-common-themes/1535
├loop5 400.80m [7:5] Empty/Unknown
│└Mounted as /dev/loop5 @ /snap/gnome-3-38-2004/112
├loop6 46.96m [7:6] Empty/Unknown
│└Mounted as /dev/loop6 @ /snap/snapd/16292
├loop7 284.00k [7:7] Empty/Unknown
│└Mounted as /dev/loop7 @ /snap/snapd-desktop-integration/14
├loop8 45.86m [7:8] Empty/Unknown
│└Mounted as /dev/loop8 @ /snap/snap-store/582
├loop9 9.09t [7:9] Empty/Unknown
│└dm-2 9.10t [253:2] Empty/Unknown
├loop10 9.09t [7:10] Empty/Unknown
│└dm-0 9.10t [253:0] Empty/Unknown
├loop11 9.09t [7:11] Empty/Unknown
│└dm-1 9.10t [253:1] Empty/Unknown
├loop12 0.00k [7:12] Empty/Unknown
└loop13 0.00k [7:13] Empty/Unknown

> Provided the data itself isn't damaged, you should be able to retrieve
> your array. Read up the section on overlays, that will protect your data
> from being overwritten while you play around trying to recover it.
>
> The other thing I would do here, is five drives is a lot to try and get
> right. I'd pick just three drives, and try to get them to assemble
> correctly (using overlays for safety!). Once you've got three drives to
> give you a functional array, you can then try and get the other two
> correct before the final recovery.
>
> What can you remember about the system? What version of superblock? And
> something I've noticed that seems weird, why do your drives have
> partitions 1 & 9, but not 2-8? That is well odd ... Have you rebuilt the
> array in any way, or is it exactly as originally created? Etc etc the
> more information you can give us the better.

I do not know the version of superblock, after the disks were
partitioned I just used "sudo mdadm --create ..." and everything went
smoothly. I didn't use any special configuration as far as I know, so
I'm not sure why the partitioning is odd. I have not rebuilt the
array, it still has the same disks as when it was created and it has
not suffered any failures.

>
> Let us know how you get on with this, and then we'll come back with more.
>
> Cheers,
> Wol

Steve




[Index of Archives]     [Linux RAID Wiki]     [ATA RAID]     [Linux SCSI Target Infrastructure]     [Linux Block]     [Linux IDE]     [Linux SCSI]     [Linux Hams]     [Device Mapper]     [Device Mapper Cryptographics]     [Kernel]     [Linux Admin]     [Linux Net]     [GFS]     [RPM]     [git]     [Yosemite Forum]


  Powered by Linux