Hi, Thanks. Please find the stderr messages snippet. We get the offsets where FIO observed failures, during the "verify" phase. The nvme**.<num>.received - is reported to contain 0. However, when we try a "dd" to access the location (offset) specified - we do see proper data. It might be an issue in our DUT, and we will investigate further. Thanks for your help. Regards, - Saju. ************************************************************* /root/demo/fio-2.12/bin/fio test.fio -eta=always --eta-newline=1 1 tee fiol.log•rite-and-verify: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16 •rite-and-verify: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16 fio-2.12 starting 1 process fio: got pattern '00', wanted '99'. Bad bits 4 fio: bad pattern block offset 0 received data dumped as nvme0n1.12288.received expected data dumped as nvme0n1.12288.expected fio: verify type mismatch (0 media, 14 given) fio: got pattern '00', wanted '99'. Bad bits 4 fio: bad pattern block offset 0 received data dumped as nvmeOn1.40960.received expected data dumped as nvme0n1.40960.expected fio: verify type mismatch (0 media, 14 given) On Tue, Dec 20, 2016 at 6:56 PM, Sitsofe Wheeler <sitsofe@xxxxxxxxx> wrote: > Hi, > > On 20 December 2016 at 12:26, Saju Nair <saju.mad.nair@xxxxxxxxx> wrote: >> >> Thanks for your clarifications. >> We ran with a --continue_on_error=verify, >> to let the FIO complete the full compare.. >> >> We tried to do a sequential write and compare, using the FIO config >> file as below, and to bring in the complexity of "random" as a 2nd >> step. >> [write-and-verify] >> rw=write >> bs=4k >> direct=1 >> ioengine=libaio >> iodepth=16 >> size=2m >> verify=pattern >> verify_pattern=0x33333333 >> continue_on_error=verify >> verify_dump=1 >> filename=/dev/XXXX >> >> FIO reports errors and we see files of the following names created: >> <filename>.<num>.received >> <filename>.<num>.expected >> >> Wanted help in interpreting the result. >> >> We wrote 2MB worth of data, with blocksize = 4K. >> So, ideally is it expected to do 2MB/4KB = 512 IO operations >> >> 1) The received/expected files: >> Are they for each 4K offset that failed the comparison ? > > I bet you can deduce this from the size and names of the files... > >> Is the <num> to be interpreted as the (num/bs)-th block that failed ? >> For ex: if the num=438272, and bs=4096 => 107th block failed ? >> >> It would be useful to know this information - so that we can debug further, >> FYI, if we try a "dd" command and check the disk, based on the above >> calculation - the data is proper (as expected). > > You never answered my question about what you are doing with stderr. > I'll repeat it here: > >>> Are you doing something like redirecting stdout to a file but not >>> doing anything with stderr? It would help if you include the command >>> line you are using to run fio in your reply. > > Can you answer this question and post the full command line you ran > fio with? I think it might have relevance to your current question. > >> 2) What were the locations that were written to.. >> Tried fio-verify-state <.state_file>, and get the below: >> Version: 0x3 >> Size: 408 >> CRC: 0x70ca464a >> Thread: 0 >> Name: write-and-verify >> Completions: 16 >> Depth: 16 >> Number IOs: 512 >> Index: 0 >> Completions: >> (file= 0) 2031616 >> (file= 0) 2035712 >> (file= 0) 2039808 >> (file= 0) 2043904 >> (file= 0) 2048000 >> (file= 0) 2052096 >> (file= 0) 2056192 >> (file= 0) 2060288 >> (file= 0) 2064384 >> (file= 0) 2068480 >> (file= 0) 2072576 >> (file= 0) 2076672 >> (file= 0) 2080768 >> (file= 0) 2084864 >> (file= 0) 2088960 >> (file= 0) 2093056 >> >> How do we interpret the above content to understand the locations of Writes. > > Perhaps fio tracks how far through the sequence it got rather than > individual locations written (this would be necessary to handle things > like loops=)? I personally don't know the answer to this but you can > always take a look at the source code. > > -- > Sitsofe | http://sucs.org/~sits/ -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html