Hi Folks,
Of course, I’m asking about this _after_ failure. It looks like it should work,
I see a commit to allow it to happen back in 2017, but something seems to be
not quite right.
One of the things I need to clarify, is this:
CONSISTENCY POLICY CHANGES
The consistency policy of an active array can be changed by using the
--consistency-policy option in Grow mode. Currently this works only
for the ppl and resync policies and allows to enable or disable the
RAID5 Partial Parity Log (PPL).
root@nas:/# mdadm --grow --consistency-policy=journal /dev/md1
mdadm: Operation not supported for consistency policy journal
That sounds like even after adding a journal device, I need to change the
policy to journal, but currently can only change to ppl and resync.
So, now I need to fix it, but seem to be stuck with no way forward.
I grabbed a copy of mdadm master from git, but it seems to be failing to
build, and I figure before pushing too hard down that path I should check
in and confirm that this really should work.
Let me know if anything else is needed, thanks.
Matt.
System: Debian buster/sid
Kernel: 4.19.2
MDADM: 4.1~rc1-4
/dev/md1:
Version : 1.2
Creation Time : Fri Jul 10 16:02:39 2015
Raid Level : raid6
Used Dev Size : 18446744073709551615
Raid Devices : 5
Total Devices : 6
Persistence : Superblock is persistent
Update Time : Wed Nov 21 17:38:59 2018
State : active, FAILED, Not Started
Active Devices : 5
Working Devices : 6
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : unknown
Name : nas:1 (local to host nas)
UUID : 4b9d2327:07ac21d0:b134c2fb:0270e37a
Events : 196099
Number Major Minor RaidDevice State
- 0 0 0 removed
- 0 0 1 removed
- 0 0 2 removed
- 0 0 3 removed
- 0 0 4 removed
- 8 1 4 sync /dev/sda1
- 259 5 - spare /dev/nvme0n1p1
- 8 67 2 sync /dev/sde3
- 8 35 0 sync /dev/sdc3
- 8 49 1 sync /dev/sdd1
- 8 17 3 sync /dev/sdb1
[ 6870.886900] md: md1 stopped.
[ 6871.096814] md/raid:md1: array cannot have both journal and bitmap
[ 6871.096816] md: pers->run() failed ...
[ 6892.528471] md/raid:md1: array cannot have both journal and bitmap
[ 6892.528474] md: pers->run() failed …
root@nas:/# mdadm -E /dev/nvme0n1p1
/dev/nvme0n1p1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x201
Array UUID : 4b9d2327:07ac21d0:b134c2fb:0270e37a
Name : nas:1 (local to host nas)
Creation Time : Fri Jul 10 16:02:39 2015
Raid Level : raid6
Raid Devices : 5
Avail Dev Size : 124735488 (59.48 GiB 63.86 GB)
Array Size : 8788666368 (8381.53 GiB 8999.59 GB)
Used Dev Size : 5859110912 (2793.84 GiB 2999.86 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=0 sectors
State : clean
Device UUID : 0b6b84ec:881c7b2c:8f12cf2d:ea527673
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Nov 21 17:38:59 2018
Bad Block Log : 512 entries available at offset 24 sectors
Checksum : 221d2dfd - correct
Events : 196099
Layout : left-symmetric
Chunk Size : 512K
Device Role : Journal
Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)
root@nas:/# mdadm -E /dev/sda1
/dev/sda1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 4b9d2327:07ac21d0:b134c2fb:0270e37a
Name : nas:1 (local to host nas)
Creation Time : Fri Jul 10 16:02:39 2015
Raid Level : raid6
Raid Devices : 5
Avail Dev Size : 5859110912 (2793.84 GiB 2999.86 GB)
Array Size : 8788666368 (8381.53 GiB 8999.59 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : clean
Device UUID : 3ab36c9f:11715fbf:383798ff:b57c3edf
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Nov 21 17:38:59 2018
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : fb48b758 - correct
Events : 196099
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)
root@nas:/# mdadm -E /dev/sdb1
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 4b9d2327:07ac21d0:b134c2fb:0270e37a
Name : nas:1 (local to host nas)
Creation Time : Fri Jul 10 16:02:39 2015
Raid Level : raid6
Raid Devices : 5
Avail Dev Size : 5859110912 (2793.84 GiB 2999.86 GB)
Array Size : 8788666368 (8381.53 GiB 8999.59 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=0 sectors
State : clean
Device UUID : 930f0646:9b6c761b:17381bd1:9a10c563
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Nov 21 17:38:59 2018
Bad Block Log : 512 entries available at offset 24 sectors
Checksum : 5402a3cf - correct
Events : 196099
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)
root@nas:/# mdadm -E /dev/sdd1
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 4b9d2327:07ac21d0:b134c2fb:0270e37a
Name : nas:1 (local to host nas)
Creation Time : Fri Jul 10 16:02:39 2015
Raid Level : raid6
Raid Devices : 5
Avail Dev Size : 5861064704 (2794.77 GiB 3000.87 GB)
Array Size : 8788666368 (8381.53 GiB 8999.59 GB)
Used Dev Size : 5859110912 (2793.84 GiB 2999.86 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=1953792 sectors
State : clean
Device UUID : 2e3ba980:bf789e60:a3903407:eaa084f0
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Nov 21 17:38:59 2018
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 96c49496 - correct
Events : 196099
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)
root@nas:/# mdadm -E /dev/sdc3
/dev/sdc3:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 4b9d2327:07ac21d0:b134c2fb:0270e37a
Name : nas:1 (local to host nas)
Creation Time : Fri Jul 10 16:02:39 2015
Raid Level : raid6
Raid Devices : 5
Avail Dev Size : 5859110912 (2793.84 GiB 2999.86 GB)
Array Size : 8788666368 (8381.53 GiB 8999.59 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : clean
Device UUID : d78f76bf:b849497e:f9d466c4:cd2648a1
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Nov 21 17:38:59 2018
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 6114b471 - correct
Events : 196099
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)
root@nas:/# mdadm -E /dev/sde3
/dev/sde3:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 4b9d2327:07ac21d0:b134c2fb:0270e37a
Name : nas:1 (local to host nas)
Creation Time : Fri Jul 10 16:02:39 2015
Raid Level : raid6
Raid Devices : 5
Avail Dev Size : 5859110912 (2793.84 GiB 2999.86 GB)
Array Size : 8788666368 (8381.53 GiB 8999.59 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=0 sectors
State : clean
Device UUID : c7db33ba:6ba9f437:d79940c5:4ececfe8
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Nov 21 17:38:59 2018
Bad Block Log : 512 entries available at offset 24 sectors
Checksum : 5ddecc49 - correct
Events : 196099
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)
root@nas:/# mdadm --assemble --force --scan -v
mdadm: looking for devices for /dev/md/1
mdadm: No super block found on /dev/dm-3 (Expected magic a92b4efc, got 46474158)
mdadm: no RAID superblock on /dev/dm-3
mdadm: No super block found on /dev/dm-1 (Expected magic a92b4efc, got 0a353538)
mdadm: no RAID superblock on /dev/dm-1
mdadm: No super block found on /dev/dm-0 (Expected magic a92b4efc, got 46474158)
mdadm: no RAID superblock on /dev/dm-0
mdadm: /dev/sde4 has wrong uuid.
mdadm: No super block found on /dev/sde2 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sde2
mdadm: No super block found on /dev/sde1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sde1
mdadm: No super block found on /dev/sde (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sde
mdadm: No super block found on /dev/md/0 (Expected magic a92b4efc, got 98a8089c)
mdadm: no RAID superblock on /dev/md/0
mdadm: /dev/sdd2 has wrong uuid.
mdadm: No super block found on /dev/sdd (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd
mdadm: /dev/sdb2 has wrong uuid.
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb
mdadm: /dev/sdc4 has wrong uuid.
mdadm: No super block found on /dev/sdc2 (Expected magic a92b4efc, got 0000003d)
mdadm: no RAID superblock on /dev/sdc2
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: /dev/sda2 has wrong uuid.
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: No super block found on /dev/nvme0n1p4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1p4
mdadm: No super block found on /dev/nvme0n1p3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1p3
mdadm: No super block found on /dev/nvme0n1p2 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1p2
mdadm: No super block found on /dev/nvme0n1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1
mdadm: /dev/sde3 is identified as a member of /dev/md/1, slot 2.
mdadm: /dev/sdd1 is identified as a member of /dev/md/1, slot 1.
mdadm: /dev/sdb1 is identified as a member of /dev/md/1, slot 3.
mdadm: /dev/sdc3 is identified as a member of /dev/md/1, slot 0.
mdadm: /dev/sda1 is identified as a member of /dev/md/1, slot 4.
mdadm: /dev/nvme0n1p1 is identified as a member of /dev/md/1, slot 65533.
mdadm: added /dev/sdd1 to /dev/md/1 as 1
mdadm: added /dev/sde3 to /dev/md/1 as 2
mdadm: added /dev/sdb1 to /dev/md/1 as 3
mdadm: added /dev/sda1 to /dev/md/1 as 4
mdadm: added /dev/nvme0n1p1 to /dev/md/1 as 65533
mdadm: added /dev/sdc3 to /dev/md/1 as 0
mdadm: failed to RUN_ARRAY /dev/md/1: Invalid argument
mdadm: looking for devices for /dev/md/0
mdadm: No super block found on /dev/dm-3 (Expected magic a92b4efc, got 46474158)
mdadm: no RAID superblock on /dev/dm-3
mdadm: No super block found on /dev/dm-1 (Expected magic a92b4efc, got 0a353538)
mdadm: no RAID superblock on /dev/dm-1
mdadm: No super block found on /dev/dm-0 (Expected magic a92b4efc, got 46474158)
mdadm: no RAID superblock on /dev/dm-0
mdadm: /dev/sde4 is busy - skipping
mdadm: /dev/sde3 has wrong uuid.
mdadm: No super block found on /dev/sde2 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sde2
mdadm: No super block found on /dev/sde1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sde1
mdadm: No super block found on /dev/sde (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sde
mdadm: No super block found on /dev/md/0 (Expected magic a92b4efc, got 98a8089c)
mdadm: no RAID superblock on /dev/md/0
mdadm: /dev/sdd2 is busy - skipping
mdadm: /dev/sdd1 has wrong uuid.
mdadm: No super block found on /dev/sdd (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdd
mdadm: /dev/sdb2 is busy - skipping
mdadm: /dev/sdb1 has wrong uuid.
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb
mdadm: /dev/sdc4 is busy - skipping
mdadm: /dev/sdc3 has wrong uuid.
mdadm: No super block found on /dev/sdc2 (Expected magic a92b4efc, got 0000003d)
mdadm: no RAID superblock on /dev/sdc2
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: /dev/sda2 is busy - skipping
mdadm: /dev/sda1 has wrong uuid.
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: No super block found on /dev/nvme0n1p4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1p4
mdadm: No super block found on /dev/nvme0n1p3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1p3
mdadm: No super block found on /dev/nvme0n1p2 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1p2
mdadm: /dev/nvme0n1p1 has wrong uuid.
mdadm: No super block found on /dev/nvme0n1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/nvme0n1
root@nas:/# mdadm -R /dev/md1
mdadm: failed to start array /dev/md/1: Invalid argument
—
Matt Wallis
mattw@xxxxxxxxxxxx
