On 09/21/2013 03:22 PM, Francis Moreau wrote: > On Fri, Sep 20, 2013 at 11:08 PM, Francis Moreau <francis.moro@xxxxxxxxx> wrote: >> Hello Martin, >> >> On Fri, Sep 20, 2013 at 8:07 PM, Martin Wilck <mwilck@xxxxxxxx> wrote: >>> On 09/20/2013 10:56 AM, Francis Moreau wrote: >>>> Hello Martin, >>>> >>>> On Mon, Sep 16, 2013 at 7:04 PM, Martin Wilck <mwilck@xxxxxxxx> wrote: >>>>> On 09/16/2013 03:56 PM, Francis Moreau wrote: >>>>> >>>>>> I did give your patch "DDF: compare_super_ddf: fix sequence number >>>>>> check" a try and now mdadm is able to detect a difference between the >>>>>> 2 disks. Therefore it refuses to insert the second disk which is >>>>>> better. >>>>>> >>>>>> However it's still not able to detect which version is the "fresher" >>>>>> like mdadm does with soft RAID1 (metadata 1.2). Therefore mdadm is not >>>>>> able to kick out the first disk if it's the outdated one. >>>>>> >>>>>> Is that expected ? >>>>> >>>>> At the moment, yes. This needs work. >>>>> >>>> >>>> Actually this is worse than I thought: with your patch applied mdadm >>>> refuses to add back a spare disk into a degraded DDF array. >>>> >>>> For example on a DDF array: >>>> >>>> # cat /proc/mdstat >>>> Personalities : [raid1] >>>> md126 : active raid1 sdb[1] sda[0] >>>> 2064384 blocks super external:/md127/0 [2/2] [UU] >>>> >>>> md127 : inactive sdb[1](S) sda[0](S) >>>> 65536 blocks super external:ddf >>>> >>>> unused devices: <none> >>>> >>>> # mdadm /dev/md126 --fail sdb >>>> [ 24.118434] md/raid1:md126: Disk failure on sdb, disabling device. >>>> [ 24.118437] md/raid1:md126: Operation continuing on 1 devices. >>>> mdadm: set sdb faulty in /dev/md126 >>>> >>>> # mdadm /dev/md127 --remove sdb >>>> mdadm: hot removed sdb from /dev/md127 >>>> >>>> # mdadm /dev/md127 --add /dev/sdb >>>> mdadm: added /dev/sdb >>>> >>>> # cat /proc/mdstat >>>> Personalities : [raid1] >>>> md126 : active raid1 sda[0] >>>> 2064384 blocks super external:/md127/0 [2/1] [U_] >>>> >>>> md127 : inactive sdb[1](S) sda[0](S) >>>> 65536 blocks super external:ddf >>>> >>>> unused devices: <none> >>>> >>>> >>>> As you can see the reinserted disk sdb sits as spare and isn't added >>>> back to the array. >>> >>> That's correct. You marked that disk failed. >>> >>>> Is it possible to add this major feature work again and keep your improvement ? >>> >>> No. A failed disk can't be added again without rebuild. I am positive >>> about that. >>> >> >> Hmm that's not the case with soft linux RAID AFAICS: doing the same >> thing with soft RAID and the reinserted disk is added to the raid >> array and it's synchronised automatically. You can try it easily. > Sorry, I didn't read your problem description carefully enough. You used mdadm --add, and that should work and should trigger a rebuild, as you said. > BTW, that's also the case for DDF if I don't apply your patch. I don't understand this. My patch doesn't change the behavior of "mdadm --add". AFAICS compare_super() isn't called in that code path. I just posted two unit tests that cover this use (or better: failure) case, please verify that they meet your scenario. On my system, with my latest patch, these tests are successful. I also tried a VM, as you suggested, and did exactly what you described, successfully. After failing/removing one disk and rebooting, the system comes up degraded; mdadm -I the old disk fails (that's correct), but I can mdadm --add the old disk and recovery starts automatically. So all is fine - the question is why it doesn't work on your system. > Additionnal information: looking at sda shows that it doesn't seem to > have metadata anymore after having added it to the container: > > # mdadm -E /dev/sda > /dev/sda: > MBR Magic : aa55 > Partition[0] : 3564382 sectors at 2048 (type 83) > Partition[1] : 559062 sectors at 3569643 (type 05) I wonder if this gives us a clue. It seems that something erased the meta data. I can't imagine that mdadm did that. I wonder if that could have been your BIOS. Pretty certainly it wasn't mdadm. However mdadm --add should work, even if the BIOS had changed something on the disk. I admit I'm clueless here. In order to make progress, we'd need mdadm -E output of both disks before and after the BIOS gets to write them, after boot, and after your trying mdadm --add. The mdmon logs would also be highly appreciated, but they'll probably hard for you to generate. You need to compile mdmon with CXFLAGS="-DDEBUG=1 -g" and make sure mdmon's stderr os captured somewhere. Regards Martin -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
