Hi everybody,
first of all, I'm using mdadm 3.3.2 on linux 4.0.1, all of my disks are
partitioned with the same geometry.
I wanted to grow my 4 disk RAID5 array to 7 disks. After adding the
disks and initiating the grow, the reshape didn't seem to start:
md0 : active raid5 sdf1[7] sde1[6] sdd1[5] sdg1[3] sdb1[4] sdh1[1]
sdc1[0]
11720044800 blocks super 1.2 level 5, 256k chunk, algorithm 2
[7/7] [UUUUUUU]
[>....................] reshape = 0.0% (0/3906681600)
finish=166847860.0min speed=0K/sec
bitmap: 0/30 pages [0KB], 65536KB chunk
I waited about three hours and checked again:
md0 : active raid5 sdf1[7] sde1[6] sdd1[5] sdg1[3] sdb1[4] sdh1[1]
sdc1[0]
11720044800 blocks super 1.2 level 5, 256k chunk, algorithm 2
[7/7] [UUUUUUU]
[>....................] reshape = 0.0% (0/3906681600)
finish=9599856140.0min speed=0K/sec
bitmap: 0/30 pages [0KB], 65536KB chunk
Unfortunately, I forgot to save the output of the grow command, but it
exited with 0.
/mdadm --misc --detail /dev/md0/ didn't show anything suspicious to me:
/dev/md0:
Version : 1.2
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Array Size : 11720044800 (11177.11 GiB 12001.33 GB)
Used Dev Size : 3906681600 (3725.70 GiB 4000.44 GB)
Raid Devices : 7
Total Devices : 7
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Mon May 11 11:55:07 2015
State : clean, reshaping
Active Devices : 7
Working Devices : 7
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 256K
Reshape Status : 0% complete
Delta Devices : 3, (4->7)
Name : anaNAS:0 (local to host anaNAS)
UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Events : 51839
Number Major Minor RaidDevice State
0 8 33 0 active sync
/dev/sdc1
1 8 113 1 active sync
/dev/sdh1
3 8 97 2 active sync
/dev/sdg1
4 8 17 3 active sync
/dev/sdb1
7 8 81 4 active sync
/dev/sdf1
6 8 65 5 active sync
/dev/sde1
5 8 49 6 active sync
/dev/sdd1
As it looked like it wouldn't be ready until long after my death and I
also wrote a backup file, somehow restarting and continuing afterwards
seemed reasonable to me.
The source I was reading suggested running /mdadm /dev/md0 --continue
--backup-file=$FILE/. Apparently this command was wrong, and I couldn't
reassamble the array:
# mdadm --assemble /dev/md0 --verbose /dev/sd[b-h]1
--backup-file=/root/grow7backup.bak
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdf1 is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sde1 is identified as a member of /dev/md0, slot 5.
mdadm: /dev/sdd1 is identified as a member of /dev/md0, slot 6.
mdadm: /dev/sdg1 is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdb1 is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdh1 is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdc1 is identified as a member of /dev/md0, slot 0.
mdadm: :/dev/md0 has an active reshape - checking if critical
section needs to be restored
mdadm: No backup metadata on /root/grow7backup.bak
mdadm: No backup metadata on device-4
mdadm: No backup metadata on device-5
mdadm: No backup metadata on device-6
mdadm: Failed to find backup of critical section
mdadm: Failed to restore critical section for reshape, sorry.
I started searching for answers but didn't find anything helpful except
the hint on the raid.wiki.kernel.org page to send an email here. The
last sentence from mdadm sounds a bit pessimistic, but I hope someone in
here can help me. The output of /mdadm --examine /dev/sd[bh]1 /is in the
attachment.
Thanks in advance,
Marco
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Name : anaNAS:0 (local to host anaNAS)
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 7813363343 (3725.70 GiB 4000.44 GB)
Array Size : 23440089600 (22354.21 GiB 24002.65 GB)
Used Dev Size : 7813363200 (3725.70 GiB 4000.44 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 721f47b6:4374452b:8cf4d3c6:672caeb3
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
Delta Devices : 3 (4->7)
Update Time : Mon May 11 14:34:33 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : c6c20176 - correct
Events : 51840
Layout : left-symmetric
Chunk Size : 256K
Device Role : Active device 3
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Name : anaNAS:0 (local to host anaNAS)
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 7813363343 (3725.70 GiB 4000.44 GB)
Array Size : 23440089600 (22354.21 GiB 24002.65 GB)
Used Dev Size : 7813363200 (3725.70 GiB 4000.44 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 528ea570:3578ca82:4e86d330:bf30634c
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
Delta Devices : 3 (4->7)
Update Time : Mon May 11 14:34:33 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : db5a0a5d - correct
Events : 51840
Layout : left-symmetric
Chunk Size : 256K
Device Role : Active device 0
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Name : anaNAS:0 (local to host anaNAS)
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 7813363343 (3725.70 GiB 4000.44 GB)
Array Size : 23440089600 (22354.21 GiB 24002.65 GB)
Used Dev Size : 7813363200 (3725.70 GiB 4000.44 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : b6dfe4a2:6db19975:36b5c400:f470ac28
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
Delta Devices : 3 (4->7)
Update Time : Mon May 11 14:34:33 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : aca3041b - correct
Events : 51840
Layout : left-symmetric
Chunk Size : 256K
Device Role : Active device 6
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Name : anaNAS:0 (local to host anaNAS)
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 7813363343 (3725.70 GiB 4000.44 GB)
Array Size : 23440089600 (22354.21 GiB 24002.65 GB)
Used Dev Size : 7813363200 (3725.70 GiB 4000.44 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : e233293b:919abecc:d5d7f0d4:7c1b748b
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
Delta Devices : 3 (4->7)
Update Time : Mon May 11 14:34:33 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : d3000e94 - correct
Events : 51840
Layout : left-symmetric
Chunk Size : 256K
Device Role : Active device 5
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Name : anaNAS:0 (local to host anaNAS)
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 7813363343 (3725.70 GiB 4000.44 GB)
Array Size : 23440089600 (22354.21 GiB 24002.65 GB)
Used Dev Size : 7813363200 (3725.70 GiB 4000.44 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 7a51397d:9785430a:bb4e502d:c07c6357
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
Delta Devices : 3 (4->7)
Update Time : Mon May 11 14:34:33 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 76e3ef5c - correct
Events : 51840
Layout : left-symmetric
Chunk Size : 256K
Device Role : Active device 4
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Name : anaNAS:0 (local to host anaNAS)
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 7813363343 (3725.70 GiB 4000.44 GB)
Array Size : 23440089600 (22354.21 GiB 24002.65 GB)
Used Dev Size : 7813363200 (3725.70 GiB 4000.44 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 7b0832a7:da8108cd:0bd25ee8:c6bc0ba6
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
Delta Devices : 3 (4->7)
Update Time : Mon May 11 14:34:33 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 6d5865f4 - correct
Events : 51840
Layout : left-symmetric
Chunk Size : 256K
Device Role : Active device 2
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : 33f0604f:46e80f5e:11b1a694:608fd9b3
Name : anaNAS:0 (local to host anaNAS)
Creation Time : Sun Nov 9 02:38:25 2014
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 7813363343 (3725.70 GiB 4000.44 GB)
Array Size : 23440089600 (22354.21 GiB 24002.65 GB)
Used Dev Size : 7813363200 (3725.70 GiB 4000.44 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : d15d44f2:1c4ed6dc:0726a874:b0422ef3
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
Delta Devices : 3 (4->7)
Update Time : Mon May 11 14:34:33 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : a1a46170 - correct
Events : 51840
Layout : left-symmetric
Chunk Size : 256K
Device Role : Active device 1
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
