On Thu, Aug 09 2018, Swapnil Ingle wrote: > Hi, > > For RAID1 with 2 slave devices underneath, we see different data on the slaves. > The IO is generated by using FIO with ioengine=libaio. > > Following script sometimes leads to have different md5sum on the slave devices. > > ================== > modprobe -r brd > modprobe brd rd_nr=2 rd_size=2097152 > dd if=/dev/zero of=/dev/ram1 oflag=direct || true > dd if=/dev/zero of=/dev/ram0 oflag=direct || true > dd if=/dev/ram1 | md5sum > dd if=/dev/ram0 | md5sum > mdadm --create md127 -f -n 2 -l raid1 /dev/ram0 /dev/ram1 <<EOF > yes > EOF > sleep 10 > cat /proc/mdstat > mkfs.ext4 /dev/md127 > mount /dev/md127 /tmp > touch /tmp/bla.txt > > echo -n " > [global] > rw=write > direct=1 > size=1G > ioengine=libaio > iodepth=128 > iodepth_batch_submit=128 > iodepth_batch_complete=128 > numjobs=4 > group_reporting > [job0] > filename=/tmp/bla.txt > [job1] > filename=/tmp/bla.txt > [job2] > filename=/tmp/bla.txt > [job3] > filename=/tmp/bla.txt > " > fio.ini > > cat fio.ini > > fio fio.ini > > ls -lah /tmp/ > > umount /tmp > > mdadm --stop md127 > > dd if=/dev/ram1 obs=1M ibs=1M skip=4 | md5sum > dd if=/dev/ram0 obs=1M ibs=1M skip=4 | md5sum > ========================== > > Is this expected behavior with async IO's? It might be expected behaviour with async direct IO. Two threads writing with O_DIRECT io to the same address could result in different data on the two devices. This doesn't seem to me to be a credible use-case though. Why would you ever want to do that in practice? NeilBrown
Attachment:
signature.asc
Description: PGP signature
