data integrity in stress tests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi fio forum,

Could you please address the following query:

I was wondering about the correct way to perform a data integrity check while using a high stress scenario,
Scenario 1: File system,  Could the following command be used for data integrity check of IO on device with FS:

fio --name=dedup --ioengine=libaio --rw=randwrite --bs=16k --numjobs=64 --thread   --direct=1  --write_iolog=C --size=400m --directory=/mnt --do_verify=1 --verify=md5

I got the message that "Multiple writers may overwrite blocks that belong to other jobs. This can cause verification failures."

In the fio man pages is written: 
filename=str      Fio normally makes up a filename based on the job name,
            thread number, and file number. If you want to share
            files between threads in a job or several jobs, specify
            a filename for each of them to override the default

Actually, fio creates 64 files and according to man pages every thread is writing to its file. Since -iodepth=1 (default) why should there be a problem with multiple writers even in the case of async ioengine?
If this is not correct, can someone suggest a data integrity check command in multithreaded fio run for devices with FS?

Scenario 2: RAW device, Could the flag --lockfile=readwrite in the following fio command solve the problem of "Multiple writers may overwrite blocks that belong to other jobs":

fio --name=dedup --ioengine=posixaio --rw=randwrite --lockfile=readwrite --bs=32k --numjobs=64 --thread --size=100m --verify=md5 --do_verify=1 --direct=1 --filename=/dev/hdiskpower2 --verify_fatal=1 --invalidate=0

In the fio man pages is written: 
Lockfile=readwrite      Read-write locking on the file. Many
                              readers may access the file at the
                              same time, but writes get exclusive
                              access.
Taking into account that that iodepth=1 and only one thread can write in a given time, one can suppose that it solves the problem of blocks overwrite since it looks similar to one threaded writer, isn't it? 
If this is not correct, can someone explain why and suggest a data integrity check command in multithreaded fio run for raw devices?


Thank you
Konstantin Ruvinsky

--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel]     [Linux SCSI]     [Linux IDE]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]

  Powered by Linux