On Sat, Oct 5, 2024 at 12:37 AM Vincent Fu <vincentfu@xxxxxxxxx> wrote:
>
> On 10/1/24 06:36, Prasad Boddupalli wrote:
> > Hello All,
> >
> > I am running fio verification in two phases:
> >
> > (1) Write phase, with do_verify=0, on the block device /dev/sdb using
> > --direct=1 flag. While the write phase is running, the disk is
> > disconnected physically. Since data got written to the block device,
> > the journal (state file) was created. After the disk was disconnected,
> > errors were in-flight writes were reported as follows:
> >
> > fio: io_u error on file /dev/sdb: Input/output error: write
> > offset=2112716800, buflen=8192
> > fio: io_u error on file /dev/sdb: Input/output error: write
> > offset=2112724992, buflen=8192
> > fio: io_u error on file /dev/sdb: Input/output error: write
> > offset=2112733184, buflen=8192
> > fio: io_u error on file /dev/sdb: Input/output error: write
> > offset=2112741376, buflen=8192
> >
> > (2) When I run the verification phase, the verification fails with the
> > following errors:
> >
> > verify: bad magic header 0, wanted acca at file /dev/sdb offset
> > 2112716800, length 8192 (requested block: offset=2112716800,
> > length=8192)
> > verify: bad magic header 0, wanted acca at file /dev/sdb offset
> > 2112724992, length 8192 (requested block: offset=2112724992,
> > length=8192)
> > verify: bad magic header 0, wanted acca at file /dev/sdb offset
> > 2112733184, length 8192 (requested block: offset=2112733184,
> > length=8192)
> >
> > The offsets reported in the errors in the verification phase
> > correspond more or less to the errors reported in the write phase.
> >
> > I am assuming that information in the state file will correspond to
> > successful writes, writes that were persisted on the disk and
> > acknowledged. If so, why was the verification done for data at offsets
> > whose writes failed in-flight ?
> >
> > Following are the options to write and verification phase:
> >
> > write:
> >
> > ' --rw=write'
> > ' --bs=8K'
> > ' --direct=1'
> > ' --ioengine=libaio'
> > ' --iodepth=16'
> > ' --verify=crc32c'
> > ' --filename={device_name}'
> > ' --size={disk_size}G'
> > ' --offset={write_offset}'
> > ' --runtime=20m'
> > ' --time_based=1'
> > ' --do_verify=0'
> >
> > verify:
> > ' --rw=read'
> > ' --bs=8K'
> > ' --direct=1'
> > ' --ioengine=libaio'
> > ' --iodepth=16'
> > ' --verify=crc32c'
> > ' --filename={device_name}'
> > ' --size={disk_size}G'
> > ' --offset={write_offset}'
> > ' --verify_only=1'
> > ' --verify_state_load=1'
> > ' --verify_dump=1'
> >
>
> When you build fio from source there is a t/fio-verify-state utility to
> display the contents of the verify state file. Are you able to run it to
> display the contents of your verify state file?
>
> Are you running on a big ending platform? This feature was broken on
> such platforms until recently.
>
> https://github.com/vincentkfu/fio-blog/wiki/Testing-on-a-big-endian-platform
Hello Vincent, thank you for your reply. I was able to run the
`fio-verify-state` tool against the state file recorded by FIO. I see
verification errors (wherein failed writes are also being verified)
only with libaio engine. When I try the same with 'psync' ioengine, I
do not see the same errors (although the throughput is greatly reduced
with 'psync' engine).
