在 2022/12/13 上午6:40, Tejun Heo 写道:
On Wed, Dec 07, 2022 at 12:38:26AM +0800, Jinke Han wrote:
From: Jinke Han <hanjinke.666@xxxxxxxxxxxxx>
Now we don't distinguish sync write ios from normal buffer write ios
in blk-throtl. A bio with REQ_SYNC tagged always mean it will be wait
until write completion soon after it submit. So it's reasonable for sync
io to complete as soon as possible.
In our test, fio writes a 100g file in sequential 4k blocksize in
a container with low bps limit configured (wbps=10M). More than 1200
ios were throttled in blk-throtl queue and the avarage throtle time
of each io is 140s. At the same time, the operation of saving a small
file by vim will be blocked amolst 140s. As a fsync will be send by vim,
the sync ios of fsync will be blocked by a huge amount of buffer write
ios ahead. This is also a priority inversion problem within one cgroup.
In the database scene, things got really bad with blk-throtle enabled
as fsync is called very often.
This patch introduces a independent sync queue for write ios and gives
a huge priority to sync write ios. I think it's a nice respond to the
semantics of REQ_SYNC. Bios with REQ_META and REQ_PRIO gains the same
priority as they are important to fs. This may avoid some potential
priority inversion problems.
I think the idea makes sense but wonder whether the implementation would be
cleaner / simpler if the sq->queued[] are indexed by SYNC, ASYNC and the
sync writes are queued in the sync queue together with reads.
Thanks.
If something is said wrong, please correct me.
If sq->queue[] were only classfied SYNC and ASYNC, some things may
become a little difficult to handle。As we put sync write and read
together into SYNC queue, the two may influence each other.
Whit wbps=1M and rbps=100M configured, sync io likely be throtled while
read ios after it may can be dispatched within the limit. In that case,
maybe we should scan the whole SYNC queue to check read io.
Thanks.
Jinke
