On Fri, May 27, 2022 at 06:52:22PM -0700, Eric Wheeler wrote: > Hi Keith, Christoph: > > Adriano who started this thread (cc'ed) reported that setting > queue/write_cache to "write back" provides much higher latency on his NVMe > than "write through"; I tested a system here and found the same thing. > > Here is Adriano's summary: > > # cat /sys/block/nvme0n1/queue/write_cache > write through > # ioping -c10 /dev/nvme0n1 -D -Y -WWW -s4K > ... > min/avg/max/mdev = 60.0 us / 78.7 us / 91.2 us / 8.20 us > ^^^^ ^^ > > # for i in /sys/block/*/queue/write_cache; do echo 'write back' > $i; done > # ioping -c10 /dev/nvme0n1 -D -Y -WWW -s4K > ... > min/avg/max/mdev = 1.81 ms / 1.89 ms / 2.01 ms / 82.3 us > ^^^^ ^^ With the "write back" setting, I find that the writes dispatched from ioping will have the force-unit-access bit set in the commands, so it is expected to take longer.