On Mon, 20 Aug 2012 11:36:10 +0200 Sergiusz Brzeziński
<Sergiusz.Brzezinski@xxxxxxxxxxxxxx> wrote:
> Hi,
>
> My system is Ubuntu 12.04, kernel 3.2.0-27. mdadm: 3.2.3
>
> If I do:
>
> # mdadm /dev/md0 -a /dev/sdc3
>
> then nothing happen! No message on command line, no info in logs!
>
> Man mdadm says:
>
> -a, --add
> hot-add listed devices. If a device appears to have recently been part of the
> array (possibly it failed or was removed) the device is re-added as describe
> in the next point. If that fails or the device was never part of the
> array, the device is added as a hot-spare. If the array is degraded, it will
> immediately start to rebuild data onto that spare.
>
> But array doesn't wont to rebuild.
>
> But not exactly. The bad info is, that sometimes it works and sometimes it doesn't.
>
> And if I restart the system, the array SOMETIMES start rebuilding and somethimes
> doesn't!
>
> There is no information what's up (comnand line or logs) so I don't know what to do.
>
>
>
> I describe bellow the whole procedure:
>
> 1.
> There ist a working Raid1 array:
>
> # cat /proc/mdstat
>
> Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4]
> [raid10]
> md0 : active raid1 sda3[2] sdb3[3]
> 115999672 blocks super 1.0 [2/2] [UU]
> bitmap: 0/1 pages [0KB], 65536KB chunk
>
> # mdadm --detail /dev/md0
>
> /dev/md0:
> Version : 1.0
> Creation Time : Wed Aug 1 15:45:56 2012
> Raid Level : raid1
> Array Size : 115999672 (110.63 GiB 118.78 GB)
> Used Dev Size : 115999672 (110.63 GiB 118.78 GB)
> Raid Devices : 2
> Total Devices : 2
> Persistence : Superblock is persistent
>
> Intent Bitmap : Internal
>
> Update Time : Mon Aug 20 11:10:00 2012
> State : active
> Active Devices : 2
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 0
>
> Name : linux:0
> UUID : b7407176:2e88c73d:2c85e940:05ff7e06
> Events : 5897
>
> Number Major Minor RaidDevice State
> 3 8 19 0 active sync /dev/sdb3
> 2 8 3 1 active sync /dev/sda3
>
>
> 2.
> I remove (phisicaly) the "/dev/sdb" drive of the box (hot-swap)
>
> # cat /proc/mdstat
>
> Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4]
> [raid10]
> md0 : active raid1 sda3[2] sdb3[3](F)
> 115999672 blocks super 1.0 [2/1] [_U]
> bitmap: 1/1 pages [4KB], 65536KB chunk
>
> # mdadm --detail /dev/md0
> /dev/md0:
> Version : 1.0
> Creation Time : Wed Aug 1 15:45:56 2012
> Raid Level : raid1
> Array Size : 115999672 (110.63 GiB 118.78 GB)
> Used Dev Size : 115999672 (110.63 GiB 118.78 GB)
> Raid Devices : 2
> Total Devices : 2
> Persistence : Superblock is persistent
>
> Intent Bitmap : Internal
>
> Update Time : Mon Aug 20 11:11:58 2012
> State : active, degraded
> Active Devices : 1
> Working Devices : 1
> Failed Devices : 1
> Spare Devices : 0
>
> Name : linux:0
> UUID : b7407176:2e88c73d:2c85e940:05ff7e06
> Events : 5908
>
> Number Major Minor RaidDevice State
> 0 0 0 0 removed
> 2 8 3 1 active sync /dev/sda3
>
> 3 8 19 - faulty spare
>
>
> 3.
> I insert the drive again - it become "/dev/sdc" instead of "/dev/sdb"
>
> 4.
> And now I do the following to rebuild the array:
>
> # mdadm /dev/md0 -a /dev/sdc3
>
> mdadm: /dev/sdc3 reports being an active member for /dev/md0, but a --re-add fails.
> mdadm: not performing --add as that would convert /dev/sdc3 in to a spare.
> mdadm: To make this a spare, use "mdadm --zero-superblock /dev/sdc3" first.
>
> I have never seen this messages before (on another systems). But this is not a
> problem. I do, what they suggest:
>
> # mdadm --zero-superblock /dev/sdc3
>
> # mdadm --examine /dev/sdc3
> mdadm: No md superblock detected on /dev/sdc3.
>
> 5.
> And now I try again rebuild the array:
>
> # mdadm /dev/md0 -a /dev/sdc3
>
> ... and nothing happen! No message, no info. I repeat some times the command.
>
> I also try this:
>
> # mdadm /dev/md0 -a -vv --force /dev/sdc3
>
> ... also nothing. Somethimes, if I repeat the command one by one, in logs appear
> the line:
>
> Aug 20 11:30:52 serwer-linmot kernel: [ 1978.111282] md: export_rdev(sdc3)
>
> ... and nothing else
>
>
> # mdadm --examine /dev/sdc3
> /dev/sdc3:
> Magic : a92b4efc
> Version : 1.0
> Feature Map : 0x1
> Array UUID : b7407176:2e88c73d:2c85e940:05ff7e06
> Name : linux:0
> Creation Time : Wed Aug 1 15:45:56 2012
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 231999344 (110.63 GiB 118.78 GB)
> Array Size : 231999344 (110.63 GiB 118.78 GB)
> Super Offset : 231999472 sectors
> State : clean
> Device UUID : f1a1f291:e10d85cc:b574c6d2:45fc0c5d
>
> Internal Bitmap : -8 sectors from superblock
> Update Time : Mon Aug 20 11:17:07 2012
> Checksum : 71bcac04 - correct
> Events : 0
>
>
> Device Role : spare
> Array State : .A ('A' == active, '.' == missing)
>
>
> # mdadm --detail /dev/md0
>
> /dev/md0:
> Version : 1.0
> Creation Time : Wed Aug 1 15:45:56 2012
> Raid Level : raid1
> Array Size : 115999672 (110.63 GiB 118.78 GB)
> Used Dev Size : 115999672 (110.63 GiB 118.78 GB)
> Raid Devices : 2
> Total Devices : 2
> Persistence : Superblock is persistent
>
> Intent Bitmap : Internal
>
> Update Time : Mon Aug 20 11:18:43 2012
> State : active, degraded
> Active Devices : 1
> Working Devices : 1
> Failed Devices : 1
> Spare Devices : 0
>
> Name : linux:0
> UUID : b7407176:2e88c73d:2c85e940:05ff7e06
> Events : 5946
>
> Number Major Minor RaidDevice State
> 0 0 0 0 removed
> 2 8 3 1 active sync /dev/sda3
>
> 3 8 19 - faulty spare
>
>
> Can anyone help?
>
I cannot see why that would happen.
It seems to be failing somewhere in bind_rdev_to_array, but I don't know
where.
Can you run
strace -o /tmp/strace mdadm /dev/md0 --add /dev/sdc3
and post the output ... or at last the tail end of the output. There should
be an ioctl which fails and I need to know what the error was (EEXIST,
EINVAL, ENOSPC are the likely ones). But don't just post that line, include
at least the last 100 lines.
NeilBrown
Attachment:
signature.asc
Description: PGP signature
