Hi Phil,
[cut]
>> Recently I bought a extra hard drive (next to existing raid level 5
>> three discs). Unfortunately during physical installation probably
>> disconnect two hard drives of existing raid on my PC. I didn't notice
>> that cables was not properly inserted. After system bootup (Linux Mint
>> 13) md doesn't start. Because /home directory should be mounted on
>> LVM@RAID my system doesn't start properly
>>
>> I've disconnected new hard drive, check and correct every cable on
>> previously working hard drives and run LiveUSB linux to check if RAID
>> will go OK. It wasn't.
>
> I wonder if you've left out some things you tried . . .
This may be significant: The step "After system bootup...my system
doesn't start properly" - I've start twice Mint in "recovery mode"
(2nd grub item). I did not mount or try to recreate array by any
command, but Linux Mint asked, if I want to try start degraded raid,
and I answer "Y"es. I don't remember exact answer, but indicated a
failure. After that I've boot from LiveUSB/CD.
>> From liveCD perspective raid 5 should be worked on three partitions:
>> /dev/sdb1
>> /dev/sdd1
>> /dev/sde1
>>
>> There are also other storage devices on PC:
>> /dev/sda - main drive, system without /home
>> /dev/sdc - LiveUSB usb
>>
>>
>> LiveUSBmint ~ # cat /proc/mdstat
>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>> [raid4] [raid10]
>> md2 : inactive sdd1[1](S) sde1[3](S)
>> 5859354352 blocks super 1.0
>>
>> unused devices: <none>
>>
>> LiveUSBmint ~ # mdadm --examine /dev/sd[bde]1
>> /dev/sdb1:
>> Magic : a92b4efc
>> Version : 1.0
>> Feature Map : 0x0
>> Array UUID : e494f7d3:bef9154e:1de134d7:476ed4e0
>> Name : tobik:2
>> Creation Time : Wed May 23 00:05:55 2012
>> Raid Level : raid5
>> Raid Devices : 3
>>
>> Avail Dev Size : 5859354352 (2793.96 GiB 2999.99 GB)
>> Array Size : 11718708480 (5587.92 GiB 5999.98 GB)
>> Used Dev Size : 5859354240 (2793.96 GiB 2999.99 GB)
>> Super Offset : 5859354608 sectors
>> State : clean
>> Device UUID : 8aa81e09:22237f15:0801f42d:95104515
>>
>> Update Time : Fri Jan 17 18:32:50 2014
>> Checksum : 8454c6e - correct
>> Events : 91
>>
>> Layout : left-symmetric
>> Chunk Size : 64K
>>
>> Device Role : Active device 0
>> Array State : AAA ('A' == active, '.' == missing)
>
> This is good.
>
>> /dev/sdd1:
>> Magic : a92b4efc
>> Version : 1.0
>> Feature Map : 0x0
>> Array UUID : e494f7d3:bef9154e:1de134d7:476ed4e0
>> Name : tobik:2
>> Creation Time : Wed May 23 00:05:55 2012
>> Raid Level : -unknown-
>> Raid Devices : 0
>>
>> Avail Dev Size : 5859354352 (2793.96 GiB 2999.99 GB)
>> Super Offset : 5859354608 sectors
>> State : active
>> Device UUID : ec85b3b8:30a31d27:6af31507:dcb4e8dc
>>
>> Update Time : Fri Jan 17 20:07:12 2014
>> Checksum : 6a2b13f4 - correct
>> Events : 1
>>
>>
>> Device Role : spare
>> Array State : ('A' == active, '.' == missing)
>
> This is bad. Simply attempting to assemble an array will not change a
> drive to a spare.
>
>> /dev/sde1:
>> Magic : a92b4efc
>> Version : 1.0
>> Feature Map : 0x0
>> Array UUID : e494f7d3:bef9154e:1de134d7:476ed4e0
>> Name : tobik:2
>> Creation Time : Wed May 23 00:05:55 2012
>> Raid Level : -unknown-
>> Raid Devices : 0
>>
>> Avail Dev Size : 5859354352 (2793.96 GiB 2999.99 GB)
>> Super Offset : 5859354608 sectors
>> State : active
>> Device UUID : 0bc9b05f:bc35f218:82798504:ef62ff32
>>
>> Update Time : Fri Jan 17 20:07:12 2014
>> Checksum : 56831dcb - correct
>> Events : 1
>>
>>
>> Device Role : spare
>> Array State : ('A' == active, '.' == missing)
>
> Same here.
>
> If the unintended disconnect was the only thing that had gone wrong,
> mdadm --assemble --force would have fixed it.
I've tried that also without success. Can't remember output (don't
have access to environment now).
> Did you try to "--add" these devices to the array while in the LiveCD?
Nope, but Linux Mint in "recovery mode" could.
>> mint etc # mdadm --examine /dev/sd[bde]1 | egrep "/dev/sd|Events|Role|Time"
>> /dev/sdb1:
>> Creation Time : Wed May 23 00:05:55 2012
>> Update Time : Fri Jan 17 18:32:50 2014
>> Events : 91
>> Device Role : Active device 0
>> /dev/sdd1:
>> Creation Time : Wed May 23 00:05:55 2012
>> Update Time : Fri Jan 17 20:07:12 2014
>> Events : 1
>> Device Role : spare
>> /dev/sde1:
>> Creation Time : Wed May 23 00:05:55 2012
>> Update Time : Fri Jan 17 20:07:12 2014
>> Events : 1
>> Device Role : spare
>>
>>
>> LiveUSBmint ~ # uname -a
>> Linux mint 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC
>> 2012 x86_64 x86_64 x86_64 GNU/Linux
>>
>> LiveUSBmint ~ # mdadm -V
>> mdadm - v3.2.5 - 18th May 2012
>>
>>
>> It is possible to recover Raid 5 from this disks? I consider
>> "Restoring array by recreating..."
>> <https://raid.wiki.kernel.org/index.php/RAID_Recovery#Restore_array_by_recreating_.28after_multiple_device_failure.29>
>> but I would like to know Your opinion. According to wiki it should be
>> considered as *last* resort.
>
> It is a last resort, but appears to be necessary in your case. There's
> only two possible device orders to choose from. Your array has version
> 1.0 metadata, so the data offset won't be a problem, but you must use
> the --size option to make sure the new array has the same size as the
> original:
>
> Try #1:
>
> mdadm --stop /dev/md2
> mdadm --create --assume-clean --metadata=1.0 --size=2929677120 \
> --chunk=64 /dev/md2 /dev/sd{b,d,e}1
>
> Show "mdadm -E /dev/sdb1" and verify that all of the sizes & offsets
> match the original.
>
> Do *not* mount the array! (Yet)
>
> Use "fsck -n" to see if the filesystem is reasonably consistent. If
> not, switch /dev/sdd1 and /dev/sde1 in try #2.
>
> When you are confortable with the device order based on "fsck -n"
> output, perform a normal fsck, then mount.
Thank you for advice. I'll try that, but first I decide to clone these
three disk (dd) and make tests on clones. I'll try to recreate array
from three hard drives. If this does not help I will try to recreate
from different sets of two drives. I'll send info after tests. Now I'm
organizing drives for clones.
>> P.S. Fortunately I have a backup, but time spend on recover can take
>> much longer.
>
> Backups are good.
I like them too :-)
Regards,
--
Mariusz Zalewski
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
