Neil Brown wrote:
On Sunday July 17, pml@xxxxxxxx wrote:
# uname -a
Linux server 2.6.12.3 #3 SMP Sun Jul 17 14:38:12 CEST 2005 i686 GNU/Linux
# ./mdadm -V
mdadm - v2.0-devel-2 - DEVELOPMENT VERSION NOT FOR REGULAR USE - 7 July 2005
...
root@server:~/dev/mdadm-2.0-devel-2# cat /proc/mdstat
Personalities : [raid5]
md1 : active raid5 sdc2[3] sdb2[1] sda2[0]
128384 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
unused devices: <none>
** mdstat mostly okay, except sdc2 is listed as device3 instead of
Hmmm, yes.... It is device number 3 in the array, but it is playing
role-2 in the raid5. When using Version-1 superblocks, we don't moved
devices around, in the "list of all devices". We just assign them
different roles. (device-N or 'spare').
So if I were to add (as an example) 7 spares to a 3 disk raid-5 array,
and later removed them for use elsewhere, a raid using a v1.x superblock
would keep a permanent listing of those drives even after being removed?
Is there a possibility (for either asthetics, or just keeping things
easier to read and possibly diagnose at a later date during manual
recoveries) of adding a command line option to "re-order and remove" old
devices that are marked as removed, that could only function if the
array was clean, and non-degraded? (this would be a manual feature we
would run, especially if automatically doing this might actually confuse
us during times of trouble-shooting?)
device2 (from 0,1,2)
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -D /dev/md1
/dev/md1:
Version : 01.00.01
Creation Time : Mon Jul 18 03:56:40 2005
Raid Level : raid5
Array Size : 128384 (125.40 MiB 131.47 MB)
Device Size : 64192 (62.70 MiB 65.73 MB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 1
Persistence : Superblock is persistent
Update Time : Mon Jul 18 03:56:42 2005
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
UUID : 1baa875e87:9ec208b2:7f5e6a27:db1f5e
Events : 1
Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/.static/dev/sda2
1 8 18 1 active sync /dev/.static/dev/sdb2
2 0 0 - removed
3 8 34 2 active sync /dev/.static/dev/sdc2
** reports version 01.00.01 superblock, but reports as if there were 4
devices used
Ok, this output definitely needs fixing. But as you can see, there
are 3 devices playing roles (RaidDevice) 0, 1, and 2. They reside in
slots 0, 1, and 3 of the array.
Depending on your answer to the first question up above, a new question
based on your comment here comes to mind... if we assume, as you say
above that it is normal for v1 superblocks to keep old removed drives
listed, but down here you say the output needs fixing, which output is
wrong in the example showing 0,1,2,3 devices, with device #2 removed,
and device 3 acting as raiddevice 2 ? If the v1 superblocks are
designed to keep removed drives listed, then the above output makes
sense.. now that you've pointed out the "feature".
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -A /dev/md1
Segmentation fault
** try to assemble the array
This is not how you assemble an array. You need to tell mdadm which
component devices to use, either on command line or in /etc/mdadm.conf
(and give --scan).
I failed to mention that I had an up to date mdadm.conf file, with the
raid UUID in it, and (I will have to verify this) I believe the command
as I typed it above, works with the 1.12 mdadm. The mdadm.conf file has
a DEVICE=/dev/hd[b-z] /dev/sd* line at the beginning of the config file,
and then the standard options (but no devices= line). Does -A still
need *some* options even if the config file is up to date?? (as I said,
I'll have to verify if 1.12 works with just the -A).
Also, if -A requires some other options on the command line, should it
not complain, instead of segfaulting? :D
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -D /dev/md1
mdadm: md device /dev/md1 does not appear to be active.
** check if its active at all
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -A /dev/md1 /dev/sda2
/dev/sdb2 /dev/sdc2
mdadm: device 1 in /dev/md1 has wrong state in superblock, but /dev/sdb2
seems ok
mdadm: device 2 in /dev/md1 has wrong state in superblock, but /dev/sdc2
seems ok
mdadm: /dev/md1 has been started with 3 drives.
** try restarting it with drive details, and it starts
Those message are a bother though. I think I know roughly what is
going on. I'll look into it shortly.
Is this possibly where the v1 superblocks are being mangled, and so it
reverts back to the v0.90 superblocks that it finds on the disk?
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -D /dev/md1
/dev/md1:
Version : 00.90.01
Creation Time : Mon Jul 18 02:53:55 2005
Raid Level : raid5
Array Size : 128384 (125.40 MiB 131.47 MB)
Device Size : 64192 (62.70 MiB 65.73 MB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 1
Persistence : Superblock is persistent
Update Time : Mon Jul 18 02:53:57 2005
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
UUID : e798f37d:baf98c2f:e714b50c:8d1018b1
Events : 0.2
Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/.static/dev/sda2
1 8 18 1 active sync /dev/.static/dev/sdb2
2 8 34 2 active sync /dev/.static/dev/sdc2
** magically, we now have a v00.90.01 superblock, it reports the proper
list of drives
Ahhh... You have assembled a different array (look at create time too).
version-1 superblocks live at a different location to version-0.90
superblocks. So it is possible to have both on the one drive. It is
supposed to pick the newest, but appears not to have done. You should
really remove old superblocks.... maybe mdadm should do that for you
???
*I* didn't assemble a different array... mdadm did ;) Yes, I agree, if
you create a *new* raid device, it should erase any form of old
superblocks, considering that it warns during creating if it detects a
drive as being part of another array, and prompts for a Y/N continue.
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -S /dev/md1
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -A /dev/md1
Segmentation fault
** try to stop and restart again, doesn't work
Again, don't do that!
Okay.. I will begin using --scan (or short form -s) from now on.. but I
*swear* <grin> that it worked without scan with the older MDADM, as long
as you had a valid DEVICE= line in the config file and possibly an ARRAY
definition also. Once again though, it shouldn't segfault, but complain
that it needs other options (and possibly list the options available
with that command).
A good example of a program that offers such insights when you mistype
or fail to provide enough options, is smartmontools.. if you type
"smartctl -t" or "smartctl -t /dev/hda" for example, leaving out the
*type* of test you wanted it to do, it will then list off the possible
test options. If you run "smartctl -t long" but forget a device name to
run the test on, it will tell you that you need to specify a device, and
gives an example.
root@server:~/dev/mdadm-2.0-devel-2# ./mdadm -E /dev/sda2
/dev/sda2:
Magic : a92b4efc
Version : 01.00
Array UUID : 1baa875e87:9ec208b2:7f5e6a27:db1f5e
Name :
Creation Time : Mon Jul 18 03:56:40 2005
Raid Level : raid5
Raid Devices : 3
Device Size : 128504 (62.76 MiB 65.79 MB)
Super Offset : 128504 sectors
State : clean
Device UUID : 1baa875e87:9ec208b2:7f5e6a27:db1f5e
Update Time : Mon Jul 18 03:56:42 2005
Checksum : 903062ed - correct
Events : 1
Layout : left-symmetric
Chunk Size : 64K
Array State : Uuu 1 failed
** the drives themselves still report a version 1 superblock... wierd
Yeh. Assemble and Examine should pick the say one by default. It
appears they don't. I'll look into it.
Thanks for the very helpful feedback.
>
NeilBrown
My pleasure Neil.. it was actually quite simple and quick testing, just
using the last little bit of space left over on 3 drives that were
slightly larger than the other 5 drives in the main array.
You can email me a patch directly, or to the list, and I can do some
more testing. I'd really like to get v1 superblocks going, but haven't
had much (reliable) luck in testing yet.
Tyler.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
