On 18/05/2021 12:54, Hannes Reinecke wrote:
In yanhui's virt workload(qemu, libaio, dio, high queue depth, single
job), the patch can improve throughput much(>50%) when running
sequential write(dio, libaio, 16 jobs) to XFS. And it is observed that
IO merge is recovered to level of disabling host tags.
I've found a good testcase for this by using null_blk:
modprobe null_blk submit_queues=32 queue_mode=2 irqmode=0 bs=4096
hw_queue_depth=2048 completion_nsec=1 nr_devices=4 shared_tags=1
shared_tag_bitmap=1
and using a simple fio job with libaio and rw=read and numjobs=32 will
do the trick:
[nullb]
rw=read
size=16g
ioengine=libaio
direct=1
buffered=0
group_reporting=1
bs=4096
numjobs=32
(of course, the 'numjobs' and 'submit_queues' parameter would need to be
adjusted to your machine).
Omitting the 'shared_tag_bitmap' module parameter would yield around 12M
iops; adding it would see a rather dramatic drop to 172k iops.
With this patchset it's back to the expected iops value.
I will consider running this test myself, but do you mean that you get
~12M with shared_tag_bitmap=1 and this patch?
I would always expect shared_tag_bitmap=1 to give a drop for null_blk,
as we move from per-submit queue driver tag to all submit queues sharing
the same driver tagset.
And I am not sure if you are fixing the IO sched from default.
Thanks,
John
