----- Original Message ----- > From: "Gi-Oh Kim" <gi-oh.kim@xxxxxxxxxxxxxxxx> > To: xni@xxxxxxxxxx > Cc: gqjiang@xxxxxxxx, "Shaohua Li" <shli@xxxxxx>, "NeilBrown" <neilb@xxxxxxx>, "Jinpu Wang" > <jinpu.wang@xxxxxxxxxxxxxxxx>, linux-raid@xxxxxxxxxxxxxxx > Sent: Friday, September 21, 2018 5:09:48 PM > Subject: Re: [RFC] wrong behavior of re-adding a device > > On Fri, Sep 21, 2018 at 7:19 AM Xiao Ni <xni@xxxxxxxxxx> wrote: > > > > > > > > ----- Original Message ----- > > > From: "Guoqing Jiang" <gqjiang@xxxxxxxx> > > > To: "Gi-Oh Kim" <gi-oh.kim@xxxxxxxxxxxxxxxx>, "Shaohua Li" <shli@xxxxxx>, > > > "NeilBrown" <neilb@xxxxxxx> > > > Cc: "Jinpu Wang" <jinpu.wang@xxxxxxxxxxxxxxxx>, > > > linux-raid@xxxxxxxxxxxxxxx > > > Sent: Friday, September 21, 2018 11:57:50 AM > > > Subject: Re: [RFC] wrong behavior of re-adding a device > > > > > > > > > > > > On 9/20/18 9:36 PM, Gi-Oh Kim wrote: > > > > On Wed, Sep 19, 2018 at 5:09 PM Gi-Oh Kim <gi-oh.kim@xxxxxxxxxxxxxxxx> > > > > wrote: > > > >> Hi, > > > >> > > > >> I found a weird behavior of re-adding a device. > > > >> I think it is a kernel bug. > > > >> I would appreciate it if somebody can confirm if it is a bug or > > > >> feature. > > > >> > > > >> I tested re-adding a device as following. > > > >> 1. create md with ram0 and ram1 > > > >> 2. add ram2 > > > >> 3. grow raid-device number to 3 > > > >> 4. remove ram2 > > > >> 5. grow raid-device number to 2 > > > >> 6. add ram2 > > > >> 7. ram0 become faulty and ram2 become active > > > >> 8. stop md > > > >> 9. assemble md with ram0 and ram1 => fail because ram0 is faulty > > > > Hi Gi > > > > I haven't reproduce your problem. Are there something wrong with the steps? > > > > [root@dell-per230-03 home]# uname -r > > 4.19.0-rc4 > > > > > > [root@dell-per230-03 home]# modprobe brd rd_nr=4 rd_size=12582912 > > [root@dell-per230-03 home]# ls /dev/ram* > > /dev/ram0 /dev/ram1 /dev/ram2 /dev/ram3 > > [root@dell-per230-03 home]# mdadm -CR /dev/md0 -l1 -n2 /dev/ram0 /dev/ram1 > > [root@dell-per230-03 home]# mdadm /dev/md0 -a /dev/ram2 > > mdadm: added /dev/ram2 > > [root@dell-per230-03 home]# cat /proc/mdstat > > Personalities : [raid1] > > md0 : active raid1 ram2[2](S) ram1[1] ram0[0] > > 12573696 blocks super 1.2 [2/2] [UU] > > [root@dell-per230-03 home]# mdadm /dev/md0 --grow --raid-device=3 > > raid_disks for /dev/md0 set to 3 > > [root@dell-per230-03 home]# cat /proc/mdstat > > Personalities : [raid1] > > md0 : active raid1 ram2[2] ram1[1] ram0[0] > > 12573696 blocks super 1.2 [3/3] [UUU] > > > > unused devices: <none> > > [root@dell-per230-03 home]# mdadm /dev/md0 -f /dev/ram2 > > mdadm: set /dev/ram2 faulty in /dev/md0 > > [root@dell-per230-03 home]# mdadm /dev/md0 -r /dev/ram2 > > mdadm: hot removed /dev/ram2 from /dev/md0 > > [root@dell-per230-03 home]# cat /proc/mdstat > > Personalities : [raid1] > > md0 : active raid1 ram1[1] ram0[0] > > 12573696 blocks super 1.2 [3/2] [UU_] > > > > unused devices: <none> > > [root@dell-per230-03 home]# mdadm /dev/md0 --grow --raid-devices=2 > > raid_disks for /dev/md0 set to 2 > > [root@dell-per230-03 home]# cat /proc/mdstat > > Personalities : [raid1] > > md0 : active raid1 ram1[1] ram0[0] > > 12573696 blocks super 1.2 [2/2] [UU] > > > > unused devices: <none> > > [root@dell-per230-03 home]# mdadm /dev/md0 -a /dev/ram2 > > mdadm: added /dev/ram2 > > Hi Xiao, > > Your mdadm shows "added" but my mdadm shows "re-added". > I think "re-add" is what cause the problem. > Could you try "mdadm /dev/md0 --re-add /dev/ram2"? Hmm, it can't be re-added. [root@dell-per230-03 mdadm]# mdadm /dev/md0 -f /dev/ram2 mdadm: set /dev/ram2 faulty in /dev/md0 [root@dell-per230-03 mdadm]# mdadm /dev/md0 -r /dev/ram2 mdadm: hot removed /dev/ram2 from /dev/md0 [root@dell-per230-03 mdadm]# mdadm /dev/md0 --grow --raid-devices=2 raid_disks for /dev/md0 set to 2 [root@dell-per230-03 mdadm]# mdadm /dev/md0 --re-add /dev/ram2 mdadm: --re-add for /dev/ram2 to /dev/md0 is not possible Regards Xiao
