> I run FIO with the following parameters and used the different engines I listed above:
> --randrepeat=1
> --ioengine=posixaio
> --direct=1
> --sync=1
> --name=mytest
> --filename=mytestfile.fio
> --overwrite=1
> --iodepth=64
> --size=100MB
> --readwrite=randrw
> --rwmixread=50
> --rwmixwrite=50
> --bs=16k
>
...
> -ioengine=psync on a 1000 IOPS Perf storage::: read=177 write=185 IOPS
> -ioengine=pvsync on a 1000 IOPS Perf storage::: read=176 write=184 IOPS
> -ioengine=sync on a 1000 IOPS Perf storage::: read=145 write=152 IOPS
> -ioengine=posixaio on a 1000 IOPS Perf storage::: read=528 write=551 IOPS*
>
> Based on the FIO results I posted above, I've concluded that the suitable FIO engine for Block Storage
> is "posixaio"; and for NFS it can be psync or pvsync or sync.
> If I use the FIO engine "posixaio" on VMware to test block storage, I'm seeing the expected IOPS.
> However, if I use the same FIO engine ("posixaio") to test NFS, I'm NOT seeing the expected IOPS. I'd
> have to use either psync, or pvsync, or sync to see the IOPS I'm expecting.
> These tests are on the same baremetal server running the version of VMWare I posted above.
>
> Can someone please shed some light on why FIO results is skewed when "posixaio" engine is used to test
> an NFS storage? The same goes with ioengines psync/pvsync/sync skews IOPS results on a block
> storage device?
NFS might introduce all sorts of problems. For the block device cases...
Synchronous engines like sync do not honor the iodepth; you need to use
jobs= to get multiple concurrent IOs. posixaio is an asynchronous engine,
so it is honoring your iodepth (it spawns threads for you).
Also, if your storage devices do any sort of buffering and caching, the
tiny 100 MiB size is likely to result in lots of cache hits, distorting
the results.
---
Robert Elliott, HPE Persistent Memory
