On 5/13/23 14:30, Eric Wheeler wrote: > On Fri, 12 May 2023, Chaitanya Kulkarni wrote: > >> Allow user to set the QUEUE_FLAG_NOWAIT optionally using module >> parameter to retain the default behaviour. Also, update respective >> allocation flags in the write path. Following are the performance >> numbers with io_uring fio engine for random read, note that device has >> been populated fully with randwrite workload before taking these >> numbers :- >> >> * linux-block (for-next) # grep IOPS bc-*fio | column -t >> >> nowait-off-1.fio: read: IOPS=482k, BW=1885MiB/s >> nowait-off-2.fio: read: IOPS=484k, BW=1889MiB/s >> nowait-off-3.fio: read: IOPS=483k, BW=1886MiB/s >> >> nowait-on-1.fio: read: IOPS=544k, BW=2125MiB/s >> nowait-on-2.fio: read: IOPS=547k, BW=2137MiB/s >> nowait-on-3.fio: read: IOPS=546k, BW=2132MiB/s >> >> * linux-block (for-next) # grep slat bc-*fio | column -t >> >> nowait-off-1.fio: slat (nsec): min=430, max=5488.5k, avg=2797.52 >> nowait-off-2.fio: slat (nsec): min=431, max=8252.4k, avg=2805.33 >> nowait-off-3.fio: slat (nsec): min=431, max=6846.6k, avg=2814.57 >> >> nowait-on-1.fio: slat (usec): min=2, max=39086, avg=87.48 >> nowait-on-2.fio: slat (usec): min=3, max=39519, avg=86.98 >> nowait-on-3.fio: slat (usec): min=3, max=38880, avg=87.17 >> >> * linux-block (for-next) # grep cpu bc-*fio | column -t >> >> nowait-off-1.fio: cpu : usr=2.77%, sys=6.57%, ctx=22015526 >> nowait-off-2.fio: cpu : usr=2.75%, sys=6.59%, ctx=22003700 >> nowait-off-3.fio: cpu : usr=2.81%, sys=6.57%, ctx=21938309 >> >> nowait-on-1.fio: cpu : usr=1.08%, sys=78.39%, ctx=2744092 >> nowait-on-2.fio: cpu : usr=1.10%, sys=79.76%, ctx=2537466 >> nowait-on-3.fio: cpu : usr=1.10%, sys=79.88%, ctx=2528092 > > Wow, amazing for such a tiny patch. Especially the latency numbers! Given > this, maybe NOWAIT should be enabled by default. yes, sending out V2 without the module parameter as my other patches has similar comment to enable nowait by default. > Why would anyone want to use the old NOWAIT=off variant? for some reason I wanted to make it backward compatible just in case this change creates regression on the setup that I don't have access to .. > Are there benefits to going without NOWAIT that go unnoticed when testing > against a ramdisk? no not at all from what I can see ramdisk enables it by default .. > For example, (and this seems unlikely) can NOWAIT affect the IO scheduler > in a way that would prevent sorted IOs headed toward a rotational disk? unfortunately I don't have a setup to run those tests :(. > > It would be interesting to see two more test classes: > > 1. Ram disk cache with NVMe backing device. > > 2. NVMe cache with rotational HDD backing device. > > -Eric > once I get a setup I'll be happy to run those numbers .. -ck
