On Sun, May 06, 2012 at 10:55:23AM +0200, Daniel wrote:
> Hello i have following Problem.
> Unfortunately through a power loss all superblocks of my RAID were
> destroyed.
That should not be possible. The superblocks are not
written in normal operation, hence power-loss should
not write them either.
I suspect something else happened here.
> My setup
>
> 8 x 1TB in a RAID 6
> 3 x 2TB in a RAID 5
>
> The crucial data are on both of these arrays.
You lost two arrays? That is even less likely with
power-loss as the cause.
Unix/Linux filesystems and raid arrays are reasonably
resistant against power-loss, as they typically run 24/7
and in many set-ups power-loss is actually the "normal"
way to take filesystem and RAID down.
> After googling a while i recognized that several users encountered that
> problem.
Can you guive some examples? I have never heard array
loss because of power loss. People overwriting their
superblocks (if at the start or 4k offset) and creating
filesystems in their raw RAID components (killing the MD
superblock depending on placement) are fairly common.
Side note: Unfortunately the current md folks have run
amock and there are three (!) different placement
optiond of the RAID superblock. The old format (0.90)
was at the end. The new format superblocks can be
at the end (1.0), start (1.1) or 4k from the start (1.2).
This is one thing that makes recovery tricky.
> Most of them were able to restore their arrays by creating the array
> again with excatly the same parameters mdadm --create .....
That is risky. If you do just one thing wrong, you may
kill other things, like a LUKS header, in the array.
You also need to specify all old values for akk the
defaults that may have changed.
For example, if you recreate an 1.0 array with version 1.1
superblock, you kill a (potential) LUKS header.
> i tried that with the 3 x 2TB Array. Unfortunately after that
> cryptsetup luksOpen tels me "md5 is no valid luks device". Then I had
> a look on the fist bytes of the array. Unfortunately it dose seem to be
> crap and no luks header. I unfortunately do not have a luks-header
> backup of this array.
> I do have a backup of the luks header of the 8 x
> 1 TB array. Bevore i Try the same with the 8 x 1 TB Array i would like
> to make deadly sure there are no other options.
Whatever you do, you can make backups before. For example,
backing up the first and last 10MB of each disk before messing
with RAID recivery, you should protect both residual superblock
data and potential LUKS headers.
In the RAID superblock dumps below, I do not find any
8 disk arrays, just 4 disk plus spaere and a dumping tool
that seems to be older than the RAID arrays, as it does
not display RAID level. I also see a mix if 0.90 and 1.2
superblocks.
For the 1.2 superblock, the raid header is at 4k offset.
The data would start after that. You hexdumped the
area before.
That all said, I think this is not a cryptsetup problem.
Better ask some md-folks what to do. With backing up the
first and last 10MB of each disk, you should be able to
protect the contents from botched RAID repair and should
be able to go back to what you had before.
However, I see a larger problem here. First, power-loss is
very, very unlikely to kill md-RAID headers even on one disk.
Then I see different header versions and the number of disks
in the headers not matching your array description. Then I
see the raid-level as -unknown- in some headers.
Care to explain the situation a bit more?
Arno
> Maby someone can Help me
> Here some information that might be helpful:
>
> Hexdump of the 3x 2TB Array
> 00000000 30 af 62 61 58 4f 88 1a a4 d2 2f ed d6 c0 98 c3
> |0.baXO..../.....|
> 00000010 1d 46 54 5f 12 38 d1 db b5 56 77 42 fb 2d df db
> |.FT_.8...VwB.-..|
> 00000020 88 6c 45 8d 2e 0c e8 39 62 88 2c b7 d1 c7 84 eb
> |.lE....9b.,.....|
> 00000030 90 25 b6 3f 0d 4c 2b 9a 38 ee 27 5a 4a 6b 75 1a
> |.%.?.L+.8.'ZJku.|
> 00000040 34 c3 58 fb 24 d1 b5 c8 ee ee ee 96 88 9a 41 c3
> |4.X.$.........A.|
> 00000050 c8 ab c1 27 3f 78 5c 06 8b 1c cc 1b f1 95 5c ba
> |...'?x\.......\.|
> 00000060 8c ce 11 e8 ba e0 8c 90 ca 5e 41 a0 d3 de 3c 4d
> |.........^A...<M|
> 00000070 4c 5d 0d 4f cc ab 06 f1 89 d2 06 04 60 29 90 da
> |L].O........`)..|
> 00000080 3d 38 50 e6 72 81 8b 18 89 92 8b 22 90 e4 e6 3c
> |=8P.r......"...<|
> 00000090 dd 66 23 8f f1 08 94 38 34 20 98 3f a9 16 9f 22 |.f#....84
> .?..."|
> 000000a0 a8 93 10 03 8f 6b 33 47 4e 31 e7 e9 e5 c3 85 87
> |.....k3GN1......|
> 000000b0 70 76 2b 68 e0 29 31 34 ee b2 e6 d9 5b be c7 96
> |pv+h.)14....[...|
> 000000c0 9d 2b 33 c1 7f 96 0b 7f f8 7e 05 0e 85 51 de ce
> |.+3......~...Q..|
> 000000d0 e7 08 3e c4 c0 0e bc 43 05 74 41 ff e3 af e0 ab
> |..>....C.tA.....|
> 000000e0 7d ce 5a 3d dd 88 d6 bb b1 88 fa 85 bd 65 73 57
> |}.Z=.........esW|
> 000000f0 f7 7c 8d 5e 37 9a 69 10 2e 30 84 a7 c6 21 72 2c
> |.|.^7.i..0...!r,|
> 00000100 51 46 80 31 79 c2 0a ec 08 04 12 a7 a6 7b 4d 3e
> |QF.1y........{M>|
> 00000110 b3 bc f7 db 20 f8 ad b6 58 c8 52 23 b9 3d 96 de |....
> ...X.R#.=..|
> 00000120 6c 6d c2 2a 8a 5d ec 3c 7c 8b d5 4f ca b8 75 5f
> |lm.*.].<|..O..u_|
> 00000130 ef b1 2c 55 cb 6a c5 f9 2f fa e9 18 fa 42 7f 32
> |..,U.j../....B.2|
> 00000140 ed e7 4a 40 71 67 c3 30 b3 e8 a0 a4 4f 46 98 59
> |..J@xxxxxxxxxx.Y|
> 00000150 43 50 74 f1 b5 42 e9 bd 90 57 59 67 fc 77 e0 d6
> |CPt..B...WYg.w..|
> 00000160 6e c5 40 56 9f 06 4d d0 e8 32 b3 2c 73 ec a1 db
> |n.@V..M..2.,s...|
> 00000170 cd d7 80 31 01 04 b2 ac 4e 3e fa ee cf a2 6a df
> |...1....N>....j.|
> 00000180 8f 74 e3 4f 8f 4a 08 8f df 75 fe 0b 35 d9 73 32
> |.t.O.J...u..5.s2|
> 00000190 37 e7 d2 71 95 b6 c4 43 f5 db cd 09 d1 ab 12 02
> |7..q...C........|
> 000001a0 bd 8f 8e 2d b0 71 6a b8 ba cf d4 44 7c 0a 34 13
> |...-.qj....D|.4.|
> 000001b0 ee 1f ec e1 c1 46 14 2c 66 94 a0 a5 7e a4 fe 0a
> |.....F.,f...~...|
> 000001c0 d7 9e 0d 9c 40 a6 57 49 f4 a0 4b b0 79 87 0b 03
> |....@.WI..K.y...|
> 000001d0 29 34 cc 79 1d 42 60 93 bc d0 e2 8d a3 20 f1 28
> |)4.y.B`...... .(|
> 000001e0 56 3d a8 a1 43 c4 fc b7 e6 8e 2c 07 f4 d3 73 01
> |V=..C.....,...s.|
> 000001f0 c4 49 1a 08 10 ac 25 0e 31 91 d2 a7 3e 85 f6 be
> |.I....%.1...>...|
> 00000200 cb e8 5f ec 1c 69 23 92 c3 84 43 aa 38 58 25 ef
> |.._..i#...C.8X%.|
> 00000210 19 90 e9 86 7a ab 47 62 f9 7b 1a 2e 09 9f 70 f5
> |....z.Gb.{....p.|
> 00000220 0a 67 9f d2 01 db ce 07 40 86 7c c4 7e cf 61 7f
> |.g......@.|.~.a.|
> 00000230 0e df a7 44 61 dc c5 56 f7 e1 14 8c 01 8e a8 0d
> |...Da..V........|
> 00000240 eb 5b 35 70 45 16 60 be 85 ce 57 9a 09 b1 82 fd
> |.[5pE.`...W.....|
> 00000250 9d 59 41 00 79 fd c0 3e e5 18 18 4f 33 9c ee 22
> |.YA.y..>...O3.."|
> 00000260 f4 2b ff 06 b2 fe 9c 0a 88 01 e6 a4 fd d5 35 01
> |.+............5.|
> 00000270 3e dd 31 84 55 5d 71 9b 73 57 94 46 72 7d c9 c0
> |>.1.U]q.sW.Fr}..|
>
> mdadm -D /dev/md5
>
> /dev/md5:
> Version : 1.2
> Creation Time : Sat May 5 01:30:36 2012
> Raid Level : raid5
> Array Size : 3907021568 (3726.03 GiB 4000.79 GB)
> Used Dev Size : 1953510784 (1863.01 GiB 2000.40 GB)
> Raid Devices : 3
> Total Devices : 3
> Persistence : Superblock is persistent
>
> Update Time : Sun May 6 07:16:52 2012
> State : clean
> Active Devices : 3
> Working Devices : 3
> Failed Devices : 0
> Spare Devices : 0
>
> Layout : left-symmetric
> Chunk Size : 128K
>
> Name : stonith:5 (local to host stonith)
> UUID : ea42de9b:87279279:b278cefd:4686698b
> Events : 21
>
> Number Major Minor RaidDevice State
> 0 8 17 0 active sync /dev/sdb1
> 1 8 49 1 active sync /dev/sdd1
> 3 8 65 2 active sync /dev/sde1
>
>
>
> mdadm -E all devices
>
> /dev/sdb:
> MBR Magic : aa55
> Partition[0] : 3907024002 sectors at 63 (type fd)
> /dev/sdb1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x0
> Array UUID : ea42de9b:87279279:b278cefd:4686698b
> Name : stonith:5 (local to host stonith)
> Creation Time : Sat May 5 01:30:36 2012
> Raid Level : raid5
> Raid Devices : 3
>
> Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
> Array Size : 7814043136 (3726.03 GiB 4000.79 GB)
> Used Dev Size : 3907021568 (1863.01 GiB 2000.40 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> State : clean
> Device UUID : 38cca227:a2b01df0:a4f71bd4:26fc69ab
>
> Update Time : Sun May 6 07:16:52 2012
> Checksum : f843636c - correct
> Events : 21
>
> Layout : left-symmetric
> Chunk Size : 128K
>
> Device Role : Active device 0
> Array State : AAA ('A' == active, '.' == missing)
>
>
>
> /dev/sdc:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type fd)
> /dev/sdc1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 22:21:42 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 5
> Preferred Minor : 0
>
> Update Time : Thu May 3 22:24:06 2012
> State : active
> Active Devices : 0
> Working Devices : 5
> Failed Devices : 0
> Spare Devices : 5
> Checksum : 829a0f47 - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 4 8 97 4 spare /dev/sdg1
>
> 0 0 8 177 0 spare /dev/sdl1
> 1 1 8 161 1 spare /dev/sdk1
> 2 2 8 145 2 spare
> 3 3 8 113 3 spare /dev/sdh1
> 4 4 8 97 4 spare /dev/sdg1
> /dev/sdd:
> MBR Magic : aa55
> Partition[0] : 3907024002 sectors at 63 (type fd)
> /dev/sdd1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x0
> Array UUID : ea42de9b:87279279:b278cefd:4686698b
> Name : stonith:5 (local to host stonith)
> Creation Time : Sat May 5 01:30:36 2012
> Raid Level : raid5
> Raid Devices : 3
>
> Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
> Array Size : 7814043136 (3726.03 GiB 4000.79 GB)
> Used Dev Size : 3907021568 (1863.01 GiB 2000.40 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> State : clean
> Device UUID : ddc9eea2:7ecb67cf:ecd45f8f:8171667b
>
> Update Time : Sun May 6 07:16:52 2012
> Checksum : de19ce91 - correct
> Events : 21
>
> Layout : left-symmetric
> Chunk Size : 128K
>
> Device Role : Active device 1
> Array State : AAA ('A' == active, '.' == missing)
> /dev/sde:
> MBR Magic : aa55
> Partition[0] : 3907024002 sectors at 63 (type fd)
> /dev/sde1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x0
> Array UUID : ea42de9b:87279279:b278cefd:4686698b
> Name : stonith:5 (local to host stonith)
> Creation Time : Sat May 5 01:30:36 2012
> Raid Level : raid5
> Raid Devices : 3
>
> Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
> Array Size : 7814043136 (3726.03 GiB 4000.79 GB)
> Used Dev Size : 3907021568 (1863.01 GiB 2000.40 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> State : clean
> Device UUID : 0e9d2be3:ab0c2188:e353ac39:f62f8178
>
> Update Time : Sun May 6 07:16:52 2012
> Checksum : 7e77205d - correct
> Events : 21
>
> Layout : left-symmetric
> Chunk Size : 128K
>
> Device Role : Active device 2
> Array State : AAA ('A' == active, '.' == missing)
> /dev/sdf:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type fd)
> /dev/sdf1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 22:21:42 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 5
> Preferred Minor : 0
>
> Update Time : Thu May 3 22:24:06 2012
> State : active
> Active Devices : 0
> Working Devices : 5
> Failed Devices : 0
> Spare Devices : 5
> Checksum : 829a0f73 - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 2 8 145 2 spare
>
> 0 0 8 177 0 spare /dev/sdl1
> 1 1 8 161 1 spare /dev/sdk1
> 2 2 8 145 2 spare
> 3 3 8 113 3 spare /dev/sdh1
> 4 4 8 97 4 spare /dev/sdg1
> /dev/sdg:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type fd)
> /dev/sdg1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 22:21:42 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 5
> Preferred Minor : 0
>
> Update Time : Thu May 3 22:24:06 2012
> State : active
> Active Devices : 0
> Working Devices : 5
> Failed Devices : 0
> Spare Devices : 5
> Checksum : 829a0f81 - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 1 8 161 1 spare /dev/sdk1
>
> 0 0 8 177 0 spare /dev/sdl1
> 1 1 8 161 1 spare /dev/sdk1
> 2 2 8 145 2 spare
> 3 3 8 113 3 spare /dev/sdh1
> 4 4 8 97 4 spare /dev/sdg1
> /dev/sdh:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type 83)
> /dev/sdh1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 22:21:42 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 5
> Preferred Minor : 0
>
> Update Time : Thu May 3 22:24:06 2012
> State : active
> Active Devices : 0
> Working Devices : 5
> Failed Devices : 0
> Spare Devices : 5
> Checksum : 829a0f55 - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 3 8 113 3 spare /dev/sdh1
>
> 0 0 8 177 0 spare /dev/sdl1
> 1 1 8 161 1 spare /dev/sdk1
> 2 2 8 145 2 spare
> 3 3 8 113 3 spare /dev/sdh1
> 4 4 8 97 4 spare /dev/sdg1
> /dev/sdi:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type fd)
> /dev/sdi1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 22:21:42 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 5
> Preferred Minor : 0
>
> Update Time : Thu May 3 22:24:06 2012
> State : active
> Active Devices : 0
> Working Devices : 5
> Failed Devices : 0
> Spare Devices : 5
> Checksum : 829a0f8f - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 0 8 177 0 spare /dev/sdl1
>
> 0 0 8 177 0 spare /dev/sdl1
> 1 1 8 161 1 spare /dev/sdk1
> 2 2 8 145 2 spare
> 3 3 8 113 3 spare /dev/sdh1
> 4 4 8 97 4 spare /dev/sdg1
> /dev/sdk:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type 83)
> /dev/sdk1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 20:50:05 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 1
> Preferred Minor : 0
>
> Update Time : Thu May 3 20:51:28 2012
> State : active
> Active Devices : 0
> Working Devices : 1
> Failed Devices : 0
> Spare Devices : 1
> Checksum : 8299e0fc - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 0 8 33 0 spare /dev/sdc1
>
> 0 0 8 33 0 spare /dev/sdc1
> /dev/sdl:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type 83)
> /dev/sdl1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 20:09:44 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 2
> Preferred Minor : 0
>
> Update Time : Thu May 3 20:10:29 2012
> State : active
> Active Devices : 0
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 2
> Checksum : 8299cdfc - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 1 8 17 1 spare /dev/sdb1
>
> 0 0 8 33 0 spare /dev/sdc1
> 1 1 8 17 1 spare /dev/sdb1
> /dev/sdm:
> MBR Magic : aa55
> Partition[0] : 1953520002 sectors at 63 (type fd)
> /dev/sdm1:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 00000000:00000000:00000000:00000000
> Creation Time : Thu May 3 20:09:44 2012
> Raid Level : -unknown-
> Raid Devices : 0
> Total Devices : 2
> Preferred Minor : 0
>
> Update Time : Thu May 3 20:10:29 2012
> State : active
> Active Devices : 0
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 2
> Checksum : 8299ce0a - correct
> Events : 1
>
>
> Number Major Minor RaidDevice State
> this 0 8 33 0 spare /dev/sdc1
>
> 0 0 8 33 0 spare /dev/sdc1
> 1 1 8 17 1 spare /dev/sdb1
>
>
> ----------------------------------------------
> Das ist ein Service von austria.com
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@xxxxxxxx
> http://www.saout.de/mailman/listinfo/dm-crypt
>
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@xxxxxxxxxxx
GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
----
One of the painful things about our time is that those who feel certainty
are stupid, and those with any imagination and understanding are filled
with doubt and indecision. -- Bertrand Russell
_______________________________________________
dm-crypt mailing list
dm-crypt@xxxxxxxx
http://www.saout.de/mailman/listinfo/dm-crypt
