Hi Daniel,
On 3/31/20 11:39 PM, Daniel Jones wrote:
Hello Phil, et al.,
Phil, after reading through your email I have some questions.
The array had bad block logging turned on. We won't re-enable this
mis-feature. It is default, so you must turn it off in your --create.
Am I able to turn off during --create? The man page for mdadm on my
system (mdadm - v4.1 - 2018-10-01) suggests that --update=no-bbl can
be used for --assemble and --manage but doesn't list it for --create.
Uhm, self compile and see what you get. In these situations, relying on
a potentially buggy system mdadm is not recommended. But if still not
available in create, fix it afterwards. You definitely do not want this.
However, the one we know appears to be the typical you'd get from
one reshape after a modern default creation (262144).
You'll have to test four combinations: all at 261124 plus one at a
time at 262144.
I'm confused by the offsets. The one remaining superblock I have
reports "Data Offset : 261120 sectors". Your email mentions 261124
and 262144. I don't understand how these three values are related?
Yeah, doing math in one's head quickly sometimes yields a fail. 262144
is 128MB in sectors. Minus 1024 sectors (your chunk size) yields
261120. /:
I think it is most likely that my one existing superblock with 261120
is one of the original three drives and not the fourth drive that was
added later. (Based on the position in drive bay).
So possible offsets (I'm still not clear on this) could be:
a) all 261120
Yes.
b) all 261124
No.
c) all 262144
No.
d) three at 261120, one at 262144
Yes.
e) three at 261120, one at 261124
f) three at 261124, one at 261120
g) three at 261124, one at 262144
No, no, and no.
h) three at 262144, one at 261120
Extremely unlikely. Not in my recommended combinations to check.
i) three at 262144, one at 261124
No.
( this ignores the combinations of not knowing which drive gets the
oddball offset )
( this also ignores for now the offsets of 259076 and 260100 mentioned below )
2) Member order for the other drives. Three drives taken three at a
time is six combinations.
3) Identity of the first drive kicked out. (Or do we know?) If not
known, there's four more combinations: whether to leave out or one of
three left out.
Can I make any tentative conclusions from this information:
Device Role : Active device 1
Array State : .AAA ('A' == active, '.' == missing, 'R' == replacing)
This device will always be listed as the 2nd member in all of your
--create commands, and always with the offset of 261120 - 2048.
I know /dev/sde is the device that didn't initially respond to BIOS
and suspect it is the "missing" drive from my superblock.
That eliminates the combinations of (3). Section (2) becomes three
drives taken two at a time (since you don't know which device role
/dev/sde had). But that is still six combinations.
I know that /dev/sdc is the drive with a working superblock that
reports itself as "Active device 1".
Right, as above.
I don't know how mdadm counts things (starting at 0 or starting at 1,
left to right or right to left, including or excluding the missing
drive).
Active devices start with zero.
Would it be reasonable for a first guess that:
.AAA = sde sdd sdc sdb (assuming the order is missing, active 0,
active 1, active 2) ?
No. Order is always active devices 0-3, with one of those replaced (in
order) with "missing".
Procedure questions:
If I understand all the above, attempted recovery is going to be along
the lines of:
mdadm --create /dev/md0 --force --assume-clean --readonly
--data-offset=261120 --chunk=512K --level=5 --raid-devices=4 missing
/dev/sdd /dev/sdc /dev/sdb
fsck -n /dev/md0
Yes, but with the order above, and with --data-offset=variable when
mixing them.
Subject to:
Don't know if --force is desirable in this case?
Not applicable to --create.
Might need to try different offsets from above. Don't know how to set
offsets if they are different per drive.
man page.
Should I start with guessing "missing" for 1 or should I start with all 4?
Might need to try all device orders.
Start by creating partitions on all devices, preferably at 2048 sectors.
(Should be the default offered.) Use data offsets 259076 and 260100
instead of 261124 and 262144.
If I understand, this is an alternative to recreating the whole-disk
mdadm containing one partition. Instead it would involve creating new
partition tables on each physical drive, creating one partition per
table, writing superblocks to the new /dev/sd[bcde]1 with offsets
adjusted by either 2044 or 2048 sectors, and then doing the fsck on
the assembled RAID.
Yes, 2048.
I think the advantage proposed here is that it prevents this
"automated superblock overwrite" from happening again if/when I try
the motherboard upgrade, but the risk I'm not comfortable with yet is
going beyond "do the minimum to get it working again". Although it
isn't practical for me to do a dd full backup of these drives, if I
can get the array mounted again I can copy off the most important data
before doing a grander repartitioning.
It's virtually impossible to correct at any time other than create, so
do it now. The "minimum" is a rather brutal situation. Fix it right.
Can you advise on any of the above?
Thanks,
DJ
Phil
