Hi,
I have a debian 8.2 (jessie) box, with a raid5 array running fine for
more than 5 years.
I tried to grew it from 6x2TB disks to 7x2TB, but the reshape got
stuck due to failures on the new added disk.
So, I bounced the system and tried to revert the reshape process.
I stopped the array: mdadm --stop /dev/md127
then try to revert the reshape process:
mdadm --assemble /dev/md127 --run --force --update=revert-reshape
/dev/sd[dcfgbe]1
But it didn't work... I got a message asking for a backup-file that
was lost after the reboot.
See below array details and devices details:
/dev/md127:
Version : 1.2
Creation Time : Tue Jul 23 23:43:47 2013
Raid Level : raid5
Used Dev Size : 1953510400 (1863.01 GiB 2000.39 GB)
Raid Devices : 6
Total Devices : 6
Persistence : Superblock is persistent
Update Time : Sat Jan 5 11:51:04 2019
State : active, Not Started
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Delta Devices : -1, (7->6)
Name : RAID
UUID : 12988b9f:aaba9653:e9c76ab3:4a5d1cc0
Events : 47758
Number Major Minor RaidDevice State
11 8 49 0 active sync /dev/sdd1
10 8 33 1 active sync /dev/sdc1
9 8 81 2 active sync /dev/sdf1
6 8 97 3 active sync /dev/sdg1
8 8 17 4 active sync /dev/sdb1
7 8 65 5 active sync /dev/sde1
---------------------------------------------------------------------------
And all devices details:
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 12988b9f:aaba9653:e9c76ab3:4a5d1cc0
Name : RAID
Creation Time : Tue Jul 23 23:43:47 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 3907025039 (1863.01 GiB 2000.40 GB)
Array Size : 9767552000 (9315.06 GiB 10001.97 GB)
Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=4239 sectors
State : clean
Device UUID : 9bee802e:f08ec230:1d2aa20d:c81c69fc
Reshape pos'n : 0
Delta Devices : -1 (7->6)
Update Time : Sat Jan 5 11:51:04 2019
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 3c9fa708 - correct
Events : 47758
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 12988b9f:aaba9653:e9c76ab3:4a5d1cc0
Name : RAID
Creation Time : Tue Jul 23 23:43:47 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 3907025039 (1863.01 GiB 2000.40 GB)
Array Size : 9767552000 (9315.06 GiB 10001.97 GB)
Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=4239 sectors
State : clean
Device UUID : 586bbd76:1ddc47ec:be08cbdf:b219606d
Reshape pos'n : 0
Delta Devices : -1 (7->6)
Update Time : Sat Jan 5 11:51:04 2019
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 83814c80 - correct
Events : 47758
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 12988b9f:aaba9653:e9c76ab3:4a5d1cc0
Name : RAID
Creation Time : Tue Jul 23 23:43:47 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 3907025039 (1863.01 GiB 2000.40 GB)
Array Size : 9767552000 (9315.06 GiB 10001.97 GB)
Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=4239 sectors
State : clean
Device UUID : a1b77982:e7acf521:e0f8f959:97f2ed4c
Reshape pos'n : 0
Delta Devices : -1 (7->6)
Update Time : Sat Jan 5 11:51:04 2019
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 1ea834a2 - correct
Events : 47758
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 12988b9f:aaba9653:e9c76ab3:4a5d1cc0
Name : RAID
Creation Time : Tue Jul 23 23:43:47 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 3907025039 (1863.01 GiB 2000.40 GB)
Array Size : 9767552000 (9315.06 GiB 10001.97 GB)
Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=4239 sectors
State : clean
Device UUID : f0ba983c:2a8c5351:a4725844:42bd9641
Reshape pos'n : 0
Delta Devices : -1 (7->6)
Update Time : Sat Jan 5 11:51:04 2019
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : e72c5996 - correct
Events : 47758
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 5
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 12988b9f:aaba9653:e9c76ab3:4a5d1cc0
Name : RAID
Creation Time : Tue Jul 23 23:43:47 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 3907025039 (1863.01 GiB 2000.40 GB)
Array Size : 9767552000 (9315.06 GiB 10001.97 GB)
Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=4239 sectors
State : clean
Device UUID : 47191b3e:33bbd598:f52ec841:98ed2a77
Reshape pos'n : 0
Delta Devices : -1 (7->6)
Update Time : Sat Jan 5 11:51:04 2019
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 6334d3a0 - correct
Events : 47758
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 12988b9f:aaba9653:e9c76ab3:4a5d1cc0
Name : RAID
Creation Time : Tue Jul 23 23:43:47 2013
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 3907025039 (1863.01 GiB 2000.40 GB)
Array Size : 9767552000 (9315.06 GiB 10001.97 GB)
Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=4239 sectors
State : clean
Device UUID : 9aa791b7:622948c4:3482db9d:85c0a8e9
Reshape pos'n : 0
Delta Devices : -1 (7->6)
Update Time : Sat Jan 5 11:51:04 2019
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : d6aef64c - correct
Events : 47758
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
---------------------------------------------------------------------------
I also checked mdadm parameters in
/sys/block/md127/md/reshape_direction: backwards
and /sys/block/md127/md/reshape_position: 0
So, I decided to reboot and after checking dmesg I saw:
md/raid:md127: reshape_position too early for auto-recovery - aborting.
md: pers->run() failed ...
I'm stuck, please, let me know how to fix this.
Thanks,
Romulo
