Hi,
Thanks..
We will certainly try with the other verify=<options> of crc32c etc.
You mention ..
>>"During a verifying read the data of the block read is kept in RAM and
if needed (depends on the verification type) the data to compare
against is regenerated."
This is what we wanted to try and exploit..
If we have a host RAM of say 64GB, then, we wanted to understand if we
could break into "N" sets of say 16GB reads and verify. ie:
Assuming that we need to verify say 160GB of data, we could do :
fio --rw=write
for i = 0 to 10 {
fio --rw=read --size=16g --verify=pattern|crc32c
--verify_pattern=<expected data>
}
Would that result in each of the FIO reads any faster ?
Regards,
- Saju.
On Tue, Dec 27, 2016 at 5:17 PM, Sitsofe Wheeler <sitsofe@xxxxxxxxx> wrote:
> Hi,
>
> On 27 December 2016 at 04:33, Saju Nair <saju.mad.nair@xxxxxxxxx> wrote:
>>
>> > Re verification speed: When you say the speed is one tenth that of
>> > regular reads are the "regular" reads also using numjobs=1? If not the
>> > comparison isn't fair and you need to rerun it with numjobs=1
>> > everywhere and tell us what the difference was for those runs.
>>
>> Yes, it was with num_jobs = 1 in both cases of "regular read" and "read-verify". I think it is understandable that there is a performance drop, since the compare/verify is done on-the-fly.. Where does FIO store the data read, before the verify step is executed.
>
> During a verifying read the data of the block read is kept in RAM and
> if needed (depends on the verification type) the data to compare
> against is regenerated. Because of this very little or no data needs
> to be stored during the write phase.
>
> One option to try and speed verification up (assuming you have the
> spare CPUs) could be to use verify_async (see the HOWTO). Another
> option could be to use faster verification comparisons (see verify=str
> of the HOWTO). For example the verify=crc32c checksum calculation will
> be hardware accelerated on recent x86 machines. Do either of these
> make a difference?
>
> --
> Sitsofe | http://sucs.org/~sits/
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
