On 09/19/2016 06:30 AM, Sunil Nadumutlu wrote:
Hi Jens, I have been observing one interesting issue since couple of days where write_verify is failing (due to data corruption) while running unaligned zoned write with bsrange ‘3k-1025K’. I have following fio CLI syntax tried on RHEL 7.0 client, by involving raw device and filesystem on raw dev as well. Here are fio CLI syntax: Raw device IO: ============== fio --runtime=43200 --filename=/dev/sdc --rw=randrw --ioengine=libaio --direct=1 --time_based --verify=md5 --verify_dump=1 --verify_fatal=1 --threads=8 --zonesize=1m --zoneskip=1024 --name=zoning-unaligned-large --bsrange=3k-1025k --runtime=86400 File system (ext4) IO: ====================== fio --runtime=43200 --filename=/tmp/xyz/fio3 --rw=randrw --ioengine=libaio --direct=1 --time_based --verify=md5 --verify_dump=1 --verify_fatal=1 --threads=8 --zonesize=1m --zoneskip=1024 --name=zoning-unaligned-large --bsrange=3k-1025k --runtime=86400 --create_on_open=1 --filesize=500m --buffer_pattern=0x48656c6c6f776f746c64 write_verify always failed within few seconds on filesystem, and write_verify failed on raw device within 2-3mins for the reason being data corruption observed during write verification. In case of raw dev, data mismatch was observed after 1K (sometiem it was across block), however incase FS, data corruption was observed across 3076 block. To narrow down the issue, I ran similar workload bs=3k ( no bsrange , and only fixed block) IO using vdbench and medusa tool, where test have passed successfully. Hence I am thinking that this is not an issue with storage array. At this time, I am just curious to know whether fio has any known bug in this area. One thing to note here is that vdbench and medusa doesn’t have block zoneing and bsrange option. It is always fixed block size with these tools. Attached herewith are dumps collected from above 2 tests… File system fio dump: fio3.1390592.expected fio3.1390592.received Raw dev fio dump: sdc.3325952.expected sdc.3325952.received Anticipating your help in this regard.
Please send questions to the fio mailing list, fio@xxxxxxxxxxxxxxx. I don't have time to answer all queries personally, plenty more people are capable of doing that.
That said, you have multiple threads writing to the same file or device in both cases.
-- Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
