On 04/24/2017 01:04 PM, Andreas Klauer wrote:
> On Mon, Apr 24, 2017 at 09:34:04AM +0200, Patrik Dahlström wrote:
> Now create two RAID sets:
>
> # losetup -D
> # for f in ? ; do cp "$f" "$f".a ; done;
> # for f in ? ; do cp "$f" "$f".b ; done;
> # for a in *.a ; do losetup --find --show "$a" ; done
> # for b in *.b ; do losetup --find --show "$b" ; done
> # mdadm --create /dev/md42 --assume-clean --level=5 --raid-devices=5 /dev/loop{0,1,2,3,4}
> # mdadm --create /dev/md42 --assume-clean --level=5 --raid-devices=6 /dev/loop{5,6,7,8,9,10}
>
> # cat /proc/mdstat
> Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
> md42 : active raid5 loop4[4] loop3[3] loop2[2] loop1[1] loop0[0]
> 405504 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
>
> md43 : active raid5 loop10[5] loop9[4] loop8[3] loop7[2] loop6[1] loop5[0]
> 506880 blocks super 1.2 level 5, 512k chunk, algorithm 2 [6/6] [UUUUUU]
>
> And compare:
>
> # hexdump -C -n 64 /dev/md42
> 00000000 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0a |000000000000000.|
> 00000010 30 30 30 30 30 30 30 30 30 30 30 30 30 31 30 0a |000000000000010.|
> 00000020 30 30 30 30 30 30 30 30 30 30 30 30 30 32 30 0a |000000000000020.|
> 00000030 30 30 30 30 30 30 30 30 30 30 30 30 30 33 30 0a |000000000000030.|
> 00000040
> # hexdump -C -n 64 /dev/md43
> 00000000 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0a |000000000000000.|
> 00000010 30 30 30 30 30 30 30 30 30 30 30 30 30 31 30 0a |000000000000010.|
> 00000020 30 30 30 30 30 30 30 30 30 30 30 30 30 32 30 0a |000000000000020.|
> 00000030 30 30 30 30 30 30 30 30 30 30 30 30 30 33 30 0a |000000000000030.|
> 00000040
>
> This is identical because in this example, the offset didn't change.
>
> # hexdump -C -n 64 -s 80808080 /dev/md42
> 04d10890 30 30 30 30 30 30 30 30 35 66 31 30 38 39 30 0a |000000005f10890.|
> 04d108a0 30 30 30 30 30 30 30 30 35 66 31 30 38 61 30 0a |000000005f108a0.|
> 04d108b0 30 30 30 30 30 30 30 30 35 66 31 30 38 62 30 0a |000000005f108b0.|
> 04d108c0 30 30 30 30 30 30 30 30 35 66 31 30 38 63 30 0a |000000005f108c0.|
> 04d108d0
> # hexdump -C -n 64 -s 80808080 /dev/md43
> 04d10890 30 30 30 30 30 30 30 30 34 64 31 30 38 39 30 0a |000000004d10890.|
> 04d108a0 30 30 30 30 30 30 30 30 34 64 31 30 38 61 30 0a |000000004d108a0.|
> 04d108b0 30 30 30 30 30 30 30 30 34 64 31 30 38 62 30 0a |000000004d108b0.|
> 04d108c0 30 30 30 30 30 30 30 30 34 64 31 30 38 63 30 0a |000000004d108c0.|
> 04d108d0
>
> For this offset, md42 was wrong, md43 is correct.
>
> # hexdump -C -n 64 -s 300808080 /dev/md42
> 11edf790 30 30 30 30 30 30 30 31 31 65 64 66 37 39 30 0a |000000011edf790.|
> 11edf7a0 30 30 30 30 30 30 30 31 31 65 64 66 37 61 30 0a |000000011edf7a0.|
> 11edf7b0 30 30 30 30 30 30 30 31 31 65 64 66 37 62 30 0a |000000011edf7b0.|
> 11edf7c0 30 30 30 30 30 30 30 31 31 65 64 66 37 63 30 0a |000000011edf7c0.|
> 11edf7d0
> # hexdump -C -n 64 -s 300808080 /dev/md43
> 11edf790 30 30 30 30 30 30 30 31 31 65 64 66 37 39 30 0a |000000011edf790.|
> 11edf7a0 30 30 30 30 30 30 30 31 31 65 64 66 37 61 30 0a |000000011edf7a0.|
> 11edf7b0 30 30 30 30 30 30 30 31 31 65 64 66 37 62 30 0a |000000011edf7b0.|
> 11edf7c0 30 30 30 30 30 30 30 31 31 65 64 66 37 63 30 0a |000000011edf7c0.|
> 11edf7d0
>
> For this offset, md42 and md43 overlapped. Grow progressed that far yet
> without writing into the original data of the 5disk raid5. This could be
> a suitable merge point for a linear device mapping.
>
> # hexdump -C -n 64 -s 400008080 /dev/md42
> 17d7a390 30 30 30 30 30 30 30 31 37 64 37 61 33 39 30 0a |000000017d7a390.|
> 17d7a3a0 30 30 30 30 30 30 30 31 37 64 37 61 33 61 30 0a |000000017d7a3a0.|
> 17d7a3b0 30 30 30 30 30 30 30 31 37 64 37 61 33 62 30 0a |000000017d7a3b0.|
> 17d7a3c0 30 30 30 30 30 30 30 31 37 64 37 61 33 63 30 0a |000000017d7a3c0.|
> 17d7a3d0
> # hexdump -C -n 64 -s 400008080 /dev/md43
> 17d7a390 30 30 30 30 30 30 30 31 33 31 37 61 33 39 30 0a |00000001317a390.|
> 17d7a3a0 30 30 30 30 30 30 30 31 33 31 37 61 33 61 30 0a |00000001317a3a0.|
> 17d7a3b0 30 30 30 30 30 30 30 31 33 31 37 61 33 62 30 0a |00000001317a3b0.|
> 17d7a3c0 30 30 30 30 30 30 30 31 33 31 37 61 33 63 30 0a |00000001317a3c0.|
> 17d7a3d0
>
> For this offset, md42 is correct and md43 is wrong.
> Grow did not progress that far.
>
> That's the general outline of the idea.
> The problem in your case is of course, your data is not that easy to verify.
I've been experimenting with this idea today by writing a small program
that looks for a mkv (matroska) header in a file, prints the offset and
then exit. I then extract 10 MB from that offset and try to play the
file with mpv.
$ ./find_matroska /dev/md1 0x202C0000000
Offset: 0x202C0000000 (2059 GB)
Offset: 0x20300000000 (2060 GB)
Found magic @ 0x2030CFCDDD7
Not matroska
Offset: 0x20340000000 (2061 GB)
Offset: 0x20380000000 (2062 GB)
Found magic @ 0x203A0005A49
Not matroska
Found magic @ 0x203AA800000
It's matroska
$ dd if=/dev/md1 bs=524288 count=20 skip=$((0x203AA800000/524288)) of=/tmp/raw.mkv
20+0 records in
20+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.0748999 s, 140 MB/s
(copy to laptop and play)
On the 6 disk raid, I had no problem finding an mkv file at relatively
low offset (< 4 TB) that would play the whole 10 MB without issues.
Somewhere between 4-5 TB I start to get corrupted videos again. I
actually stumbled upon the exact same video on offsets:
0x400C1800000 (4 TB) : Intact, grow has come here
0x500F2B00000 (5 TB) : Corrupted, grow did not progress here yet
However, I was not able to do that on the 5 disk raid. It didn't matter
if I started the search at 1 TB or 16 TB, it would always have errors.
Would this mean that my data offset is wrong for the 5 disk raid?
Best regards
// Patrik
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
