On 07/24/2015 01:49 PM, Roman Pen wrote:
fio does not provide any possibility to verify checksum of a block with meta
information inside. You can create configuration for verifincation checksum
of random data either you can verify meta information with some pattern or
random data, but not both.
Why checksumming and meta together can be useful? Meta helps to figure out internally
on filesystem or storage what block was written in case of corruption, i.e. offset
of the block and block number explicitly tell us the virtual address of the block.
On the other hand checksum of random data helps to detect corruption. Using meta
and pattern together do not help a lot, since 'verify_interval' can be big enough
and same sequence of pattern bytes will be undistinguishable internally on filesystem
or storage.
Also, it seems to me that keeping meta header separately from generic verify header
does not make a lot of sense, since generic verify header can include all members
of meta header without any performance or other impact.
In this patch I move all members from vhdr_meta structure to generic verify_header,
always verifying meta with the possiblity to checksum the following data: random
or pattern.
You are allowed to specify verify_pattern=str with any of the possible verification
methods and have also meta verification, i.e.
verify=md5
verify_pattern=0xfe
or
verify=sha1
verify_pattern=0xff
etc.
To keep everything compatible with old configurations it is still possible to specify
verify=meta
but this option marked and depricated and kept only for compatibility reasons.
Before that patch the verification layout according to the specified options looks
as the following, e.g.:
#1
--
verify=meta
verify_pattern=0xff
--
result layout of each block: [hdr|meta|pattern]
#2
--
verify_pattern=0xff
--
result layout of each block: [hdr|pattern]
#3
--
verify=pattern
verify_pattern=0xff
--
result layout of each block: [pattern]
After applying of the patch 'vhdr_meta' is always embedded into 'verify_header' and layout
looks as the following, e.g.:
#1
--
verify=meta
verify_pattern=0xff
--
result layout of each block: [hdr+meta|pattern]
#2
--
verify=md5|sha1|etc
verify_pattern=0xff
--
result layout of each block: [hdr+meta|cksum|pattern]
#3
--
verify_pattern=0xff
--
result layout of each block: [hdr+meta|pattern]
#4
--
verify=pattern
verify_pattern=0xff
--
result layout of each block: [pattern]
Good work! I'll make some time this week to fully review this, so we can
get it committed.
If you feel like you are up for more in this area, I'd love to have a
mode that writes verify info based the location on disk/file, and hence
doesn't need a header. This could/should work with any of the hashes,
and it should have a way to specify the seed for the data. The generated
data for the block would be a function of the offset, hash, seed.
--
Jens Axboe
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
