Hi, I have a 4 drive imsm RAID 5 array which is working fine. I want to remove one of the drives, sda, and replace it with a spare, sdc. From man mdadm I understand that add - fail - remove is the way to go but this does not work.
Before:
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md124 : active raid5 sdd[3] sdb[2] sda[1] sde[0]
2831155200 blocks super external:/md126/0 level 5,
128k chunk, algorithm 0 [4/4] [UUUU]
md125 : active raid5 sdd[3] sdb[2] sda[1] sde[0]
99116032 blocks super external:/md126/1 level 5, 1
28k chunk, algorithm 0 [4/4] [UUUU]
md126 : inactive sda[3](S) sdb[2](S) sdd[1](S) sde[0](S)
14681 blocks super external:imsm
unused devices: <none>
I can add (or add-spare) which increases the size of the container and though I can't see any spare drives listed by mdadm, it appears as SPARE DISK in the Intel option ROM after a reboot.
$ sudo mdadm --zero-superblock /dev/sdc
$ sudo mdadm /dev/md/imsm1 --add-spare /de
v/sdc
mdadm: added /dev/sdc
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md124 : active raid5 sdd[3] sdb[2] sda[1] sde[0]
2831155200 blocks super external:/md126/0 level 5,
128k chunk, algorithm 0 [4/4] [UUUU]
md125 : active raid5 sdd[3] sdb[2] sda[1] sde[0]
99116032 blocks super external:/md126/1 level 5, 1
28k chunk, algorithm 0 [4/4] [UUUU]
md126 : inactive sdc[4](S) sda[3](S) sdb[2](S) sdd[1](S) sde[0](S)
15786 blocks super external:imsm
unused devices: <none>
$
No spare devices listed here:
$ sudo mdadm -D /dev/md/imsm1
/dev/md/imsm1:
Version : imsm
Raid Level : container
Total Devices : 5
Working Devices : 5
UUID : bdb7f495:21b8c189:e496c216:6f2d6c4c
Member Arrays : /dev/md/md1_0 /dev/md/md0_0
Number Major Minor RaidDevice
- 8 64 - /dev/sde
- 8 32 - /dev/sdc
- 8 0 - /dev/sda
- 8 48 - /dev/sdd
- 8 16 - /dev/sdb
$
Trying to remove sda fails.
$ sudo mdadm --fail /dev/md126 /dev/sda
mdadm: Cannot remove /dev/sda from /dev/md126, array will be failed.
sda is 2TB, the others are 1TB - is that a problem?
smartctl shows 2 drives don't support SCT and it's disabled on the other 3.
There's a very similar question here from Edwin in 2017:
https://unix.stackexchange.com/questions/372908/add-hot-spare-drive-to-intel-rst-onboard-raid#372920
The only reply points to an Intel doc which uses the standard command to add a drive but doesn't show the result.
$ uname -a
Linux Intel 6.9.2-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 26
May 2024 01:30:29 +0000 x86_64 GNU/Linux
$ mdadm --version
mdadm - v4.3 - 2024-02-15
