On Sun, 06 Oct 2024 07:00:18 +0100 19 Devices <19devices@xxxxxxxxx> wrote: > Hi, I have a 4 drive imsm RAID 5 array which is working fine. I want to > remove one of the drives, sda, and replace it with a spare, sdc. From man > mdadm I understand that add - fail - remove is the way to go but this does > not work. > > Before: > $ cat /proc/mdstat > Personalities : [raid6] [raid5] [raid4] > md124 : active raid5 sdd[3] sdb[2] sda[1] sde[0] > 2831155200 blocks super external:/md126/0 level 5, > 128k chunk, algorithm 0 [4/4] [UUUU] > > md125 : active raid5 sdd[3] sdb[2] sda[1] sde[0] > 99116032 blocks super external:/md126/1 level 5, 1 > 28k chunk, algorithm 0 [4/4] [UUUU] > > md126 : inactive sda[3](S) sdb[2](S) sdd[1](S) sde[0](S) > 14681 blocks super external:imsm > > unused devices: <none> > > > I can add (or add-spare) which increases the size of the container and though > I can't see any spare drives listed by mdadm, it appears as SPARE DISK in the > Intel option ROM after a reboot. > > $ sudo mdadm --zero-superblock /dev/sdc > > $ sudo mdadm /dev/md/imsm1 --add-spare /de > v/sdc > mdadm: added /dev/sdc > > $ cat /proc/mdstat > Personalities : [raid6] [raid5] [raid4] > md124 : active raid5 sdd[3] sdb[2] sda[1] sde[0] > 2831155200 blocks super external:/md126/0 level 5, > 128k chunk, algorithm 0 [4/4] [UUUU] > > md125 : active raid5 sdd[3] sdb[2] sda[1] sde[0] > 99116032 blocks super external:/md126/1 level 5, 1 > 28k chunk, algorithm 0 [4/4] [UUUU] > > md126 : inactive sdc[4](S) sda[3](S) sdb[2](S) sdd[1](S) sde[0](S) > 15786 blocks super external:imsm > > unused devices: <none> > $ > > > No spare devices listed here: > > $ sudo mdadm -D /dev/md/imsm1 > /dev/md/imsm1: > Version : imsm > Raid Level : container > Total Devices : 5 > > Working Devices : 5 > > > UUID : bdb7f495:21b8c189:e496c216:6f2d6c4c > Member Arrays : /dev/md/md1_0 /dev/md/md0_0 > > Number Major Minor RaidDevice > > - 8 64 - /dev/sde > - 8 32 - /dev/sdc > - 8 0 - /dev/sda > - 8 48 - /dev/sdd > - 8 16 - /dev/sdb > $ > Hello, I know. It is fine. From container point of view these all are spares. Nobody ever complained about that so we did not fixed it :) The most important is that all drives are here. To detect spares you must compare this list with list from #mdadm --detail /dev/md124 (member array). Drives that are not used in member array are spares. > > Trying to remove sda fails. > > $ sudo mdadm --fail /dev/md126 /dev/sda > mdadm: Cannot remove /dev/sda from /dev/md126, array will be failed. It might be an issue in mdadm, we added this and later we added fixes: Commit: https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/commit/?id=fc6fd4063769f4194c3fb8f77b32b2819e140fb9 Fixes: https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/commit/?id=b3e7b7eb1dfedd7cbd9a3800e884941f67d94c96 https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/commit/?id=461fae7e7809670d286cc19aac5bfa861c29f93a but your release is mdadm-4.3, all fixes should be there. It might be a new bug. Try: #mdadm -If sda but please do not abuse it (just use it one time because it may fail your array). According to mdstat it should be safe in this case. If you can do some investigation, I would be tankful, I expect issues in enough() function. Thanks, Mariusz > > sda is 2TB, the others are 1TB - is that a problem? > > smartctl shows 2 drives don't support SCT and it's disabled on the other 3. > > There's a very similar question here from Edwin in 2017: > https://unix.stackexchange.com/questions/372908/add-hot-spare-drive-to-intel-rst-onboard-raid#372920 > > The only reply points to an Intel doc which uses the standard command to add > a drive but doesn't show the result. > > $ uname -a > Linux Intel 6.9.2-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 26 > May 2024 01:30:29 +0000 x86_64 GNU/Linux > > $ mdadm --version > mdadm - v4.3 - 2024-02-15 >
