On 04/06/2019 00:16, Robby Nelson wrote: > All, > First I simulated corruption and repair using a raid of two .img files in a similar manner as here: > https://gist.github.com/MawKKe/caa2bbf7edcc072129d73b61ae7815fb > > I am having issues doing the same on my actual disks and am curious if I have configured something incorrectly. The disks are 4TB each and so I started out by testing using 50GB partitions, otherwise cryptsetup takes about 16 hours and then another 16 hours to assemble the raid. These are the steps I took: > > * Create a 50G partition on each disk, /dev/sdb1 and /dev/sdd1 > * Encrypt each partition, cryptsetup -v --type luks2 --key-size 512 --sector-size 4096 --integrity hmac-sha256 luksFormat /dev/sdXX /path/to/keyfile > * Open devices, cryptsetup open --type luks cryptraidY /dev/sdXX --key-file /path/to/keyfile > * Create raid, mdadm --create --verbose --level=10 --metadata=1.2 --chunk=512 --raid-devices=2 --layout=f2 /dev/md/cryptraid /dev/mapper/cryptraid0 /dev/mapper/cryptraid1 > * Create filesystem, mkfs.xfs /dev/md/cryptraid > * Scrub array, echo check > /sys/block/mdX/md/sync_action > * dmesg shows, md: data-check of RAID array mdX / md: mdX data-check done > * If I then try to corrupt a disk using dd, dd if=/dev/random of=/dev/sdd1 seek=1000 bs=1M count=1, then scrub, no errors are shown. > > > The only way I have been able to get the scrub to reveal errors if by doing one of the following: > > * creating a file on the fs when mounted (seems like this should not), dd if=/dev/random of=~/Test/testfile bs=1M count=1 > * stop the raid, use dd on a partition as above (dd if=/dev/random of=/dev/sdd1 seek=1000 bs=1M count=1), assemble the raid, then scrub > > Is this the expected behavior? Perhaps I am incorrectly using dd to simulate corruption? Hi, Perhaps, this looks like some storage stack still uses cached data (page cache). Try to repeat with dd using direct-io (add oflag=direct) and flush all caches immediately after you simulated data corruption "echo 3 > /proc/sys/vm/drop_caches" (as root) Anyway, better list to discuss this is dm-devel@xxxxxxxxxx (device-mapper development). Also, I know there are various bugs in MD Raid code that are being discovered and fixed when using in combination with dm-integrity (proper people should be on dm-devel as well). Milan _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx https://www.saout.de/mailman/listinfo/dm-crypt
