On 5/22/19 8:27 PM, Song Liu wrote:
> On Wed, May 22, 2019 at 6:18 AM Michal Soltys <soltys@xxxxxxxx> wrote:
>>
>> Hi,
>>
>> As I started experimenting with journaled raids:
>>
>> 1) can another (raid1) device or lvm mirror (using md underneath) be used formally as a write journal ?
>> 2) for the testing purposes, are loopback devices expected to work ?
>>
>> I know I can successfully create both of the above (case scenario below), but any attempt to write to such md device ends with hung tasks and D states.
>>
>>
>> kernel 5.1.3
>> mdadm 4.1
>> btrfs filesystem with /var subvolume (where losetup files were created with fallocate)
>>
>> What I did:
>>
>> - six 10gb files as backings for /dev/loop[0-5]
>> - /dev/loop[4-5] - raid1
>> - /dev/loop[0-3] - raid5
>>
>> Now at this point both raids work just fine, separately. The problem starts if I create raid1 as raid5's journal (doesn't matter whether w-t or w-b). Any attempt to write to a device like that instantly ends with D and respective traces in dmesg:
>
> Hi Michal,
>
> Yes, the journal _should_ work with another md, lvm, or loop device.
> Could you please share the commands you used in this test?
>
> Thanks,
> Song
>
Actually, this seems to be unreleated to underlying devices - the culprit seems to be attempting to write to an array after adding journal, without stopping and reassembling it first. Details below.
0) Common start
# cd /var/tmp
# for x in {0..5}; do fallocate -l 10g $x; done
# for x in {0..5}; do losetup /dev/loop$x $x; done
1) Following from (0), *works*:
# mdadm -C /dev/md/journ -N journ -n2 -l1 -e1.2 /dev/loop[45]
(wait for sync / or create with --assume-clean)
# mdadm -C /dev/md/master -N master -n4 -l5 -e1.2 --write-journal /dev/md/journ /dev/loop[0123]
(wait for sync / or create with --assume-clean)
# mdadm -Es >>/dev/mdadm.conf
# dd if=/dev/zero of=/dev/md/master bs=262144 count=1
(works)
# mdadm -Ss
# mdadm -A /dev/md/journ
# mdadm -A /dev/md/master
# dd if=/dev/zero of=/dev/md/master bs=262144 count=1
(works)
2) Following from (0), *does not* work (adding journal afterwards):
# mdadm -C /dev/md/journ -N journ -n2 -l1 -e1.2 /dev/loop[45]
(wait for sync / or create with --assume-clean)
# mdadm -C /dev/md/master -N master -n4 -l5 -e1.2 /dev/loop[0123]
(wait for sync / or create with --assume-clean)
# mdadm --readonly /dev/md/master
# mdadm /dev/md/master --add-journal /dev/md/journ
# dd if=/dev/zero of=/dev/md/master bs=262144 count=1
(hangs)
Variation of the above that does work:
- adding journal afterwards
- stopping both arrays
- assembly
- writing
3) Following from (0), *does not* work (adding correct journal after assembly with missing one):
mdadm -C /dev/md/journ -N journ -n2 -l1 -e1.2 /dev/loop[45]
(wait for sync / or create with --assume-clean)
mdadm -C /dev/md/master -N master -n4 -l5 -e1.2 --write-journal /dev/md/journ /dev/loop[0123]
(wait for sync / or create with --assume-clean)
mdadm -Es >>/dev/mdadm.conf
mdadm -Ss
# mdadm -A --force /dev/md/master
mdadm: Journal is missing or stale, starting array read only.
mdadm: /dev/md/master has been started with 4 drives.
# mdadm -A /dev/md/journ
mdadm: /dev/md/journ has been started with 2 drives.
# mdadm /dev/md/master --add-journal /dev/md/journ
mdadm: Journal added successfully, making /dev/md/master read-write
mdadm: added /dev/md/journ
# dd if=/dev/zero of=/dev/md/master bs=262144 count=1
(hangs)
Same variation as in (2) - stopping and assembling normally - works as well.
