Re: Unable to restart reshape

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




Zitat von Michael Busby <michael.a.busby@xxxxxxxxx>:

I have a system the was doing a reshape from RAID5 to 6, the system
had to be powered off this morning and moved, upon restarting the
server i issued the following command to continue the reshape

 mdadm -A /dev/md0 --backup-file=/home/md.backup

i get back to following error

mdadm: Failed to restore critical section for reshape, sorry.

any idea why?

before shutting down cat /proc/mdstat showed

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : active raid6 sdf[0] sdb[6](S) sda[4] sdc[3] sde[2] sdd[1]
    7814055936 blocks super 1.0 level 6, 512k chunk, algorithm 18
[6/5] [UUUUU_]
    [==============>......]  reshape = 70.8% (1384415232/1953513984)
finish=3658.6min speed=2592K/sec

but now it shows

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : inactive sdc[3] sdb[6](S) sde[2] sdd[1] sdf[0]
     9767572240 blocks super 1.0

i am totally confused, it seems to have lost a drive from the raid,
and the number of blocks is incorrect


issuing the following

 mdadm -Avv --backup-file=/home/md.backup /dev/md0

returns


mdadm: looking for devices for /dev/md0
mdadm: cannot open device /dev/sda5: Device or resource busy
mdadm: /dev/sda5 has wrong uuid.
mdadm: no RAID superblock on /dev/sda2
mdadm: /dev/sda2 has wrong uuid.
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: /dev/sda1 has wrong uuid.
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sda has wrong uuid.
mdadm: /dev/sdg is identified as a member of /dev/md0, slot -1.
mdadm: /dev/sdf is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot 3.
mdadm:/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: backup-metadata found on /home/md.backup but is not needed
mdadm: Failed to find backup of critical section
mdadm: Failed to restore critical section for reshape, sorry.


seem the above was trying at use the wrong disks to assemble, so using
the following

mdadm -Avv /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]

 mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
mdadm:/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: backup-metadata found on /home/md.backup but is not needed
mdadm: Failed to find backup of critical section
mdadm: Failed to restore critical section for reshape, sorry.


have now upgraded to mdadm 3.2.2

and get a little more info

mdadm -Avv /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]

mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
mdadm: device 6 in /dev/md0 has wrong state in superblock, but /dev/sdb seems ok
mdadm:/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: backup-metadata found on /home/md.backup but is not needed
mdadm: Failed to find backup of critical section
mdadm: Failed to restore critical section for reshape, sorry.



Ok, i dont know if this is the right thing to have done

~# mdadm -Avv --force /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]

mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
mdadm: clearing FAULTY flag for device 1 in /dev/md0 for /dev/sdb
mdadm: Marking array /dev/md0 as 'clean'
mdadm:/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: backup-metadata found on /home/md.backup but is not needed
mdadm: Failed to find backup of critical section
mdadm: Failed to restore critical section for reshape, sorry.


~# mdadm -Avv /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]

mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
mdadm:/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: restoring critical section
mdadm: added /dev/sdd to /dev/md0 as 1
mdadm: added /dev/sde to /dev/md0 as 2
mdadm: added /dev/sdc to /dev/md0 as 3
mdadm: added /dev/sda to /dev/md0 as 4
mdadm: no uptodate device for slot 5 of /dev/md0
mdadm: added /dev/sdb to /dev/md0 as -1
mdadm: added /dev/sdf to /dev/md0 as 0
mdadm: /dev/md0 has been started with 4 drives (out of 6) and 1 spare.

~# cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : active raid6 sdf[0] sdb[6](S) sdc[3] sde[2] sdd[1]
      7814055936 blocks super 1.0 level 6, 512k chunk, algorithm 18
[6/4] [UUUU__]
      [==============>......]  reshape = 74.3% (1452929024/1953513984)
finish=2545.2min speed=3276K/sec

unused devices: <none>

so looks like its carrying on now but with 4 disks and a spare, maybe
i can add the other disk once the reshape has finished

It generally helps to include/examine "mdadm -E /dev/sdX" of all devices involved in your mail(s) and also "mdadm -Q --detail /dev/md0". After the reshape is done it will automatically rebuild using the spare. Then you can have a close look which of your devices arent used, clear the metadate from the device and add it as well to regain full redundancy. You'll have plenty hours of fun watching /proc/mdstat. ;)
Alex.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux RAID Wiki]     [ATA RAID]     [Linux SCSI Target Infrastructure]     [Linux Block]     [Linux IDE]     [Linux SCSI]     [Linux Hams]     [Device Mapper]     [Device Mapper Cryptographics]     [Kernel]     [Linux Admin]     [Linux Net]     [GFS]     [RPM]     [git]     [Yosemite Forum]


  Powered by Linux