> Il giorno 25 ago 2021, alle ore 18:43, Jan Kara <jack@xxxxxxx> ha scritto:
>
> On Mon 23-08-21 18:06:18, Jan Kara wrote:
>> On Mon 23-08-21 15:58:25, Paolo Valente wrote:
>>>> Currently I'm running wider set of benchmarks for the patches to see
>>>> whether I didn't regress anything else. If not, I'll post the patches to
>>>> the list.
>>>
>>> Any news?
>>
>> It took a while for all those benchmarks to run. Overall results look sane,
>> I'm just verifying by hand now whether some of the localized regressions
>> (usually specific to a particular fs+machine config) are due to a measurement
>> noise or real regressions...
>
> OK, so after some manual analysis I've found out that dbench indeed becomes
> more noisy with my changes for high numbers of processes. I'm leaving for
> vacation soon so I will not be probably able to debug it before I leave but
> let me ask you one thing: The problematic change seems to be mostly a
> revert of 7cc4ffc55564 ("block, bfq: put reqs of waker and woken in
> dispatch list") and I'm currently puzzled why it has such an effect. What
> I've found out is that 7cc4ffc55564 results in IO of higher priority
> process being injected into the time slice of lower priority process and
> thus there's always only single busy queue (of the lower priority process)
> and thus higher priority process queue never gets scheduled. As a result
> higher priority IO always competes with lower priority IO and there's no
> service differentiation (we get 50/50 split of throughput between the
> processes despite different IO priorities).
I need a little help here. Since high-priority I/O is immediately
injected, I wonder why it does not receive all the bandwidth it
demands. Maybe, from your analysis, you have an answer. Perhaps it
happens because:
1) high-priority I/O is FIFO-queued with lower-priority I/O in the
dispatch list?
or
2) immediate injection prevents idling from being performed in favor
of high-priority I/O?
> And this scenario shows that
> always injecting IO of waker/wakee isn't desirable, especially in a way as
> done in 7cc4ffc55564 where injected IO isn't accounted within BFQ at all
> (which easily allows for service degradation unnoticed by BFQ).
Not sure that accounting would help high-priority I/O in your scenario.
> That's why
> I've basically reverted that commit on the ground that on next dispatch we
> call bfq_select_queue() which will see waker/wakee has IO to do and can
> decide to inject the IO anyway. We do more CPU work but the IO pattern
> should be similar. But apparently I was wrong :)
For the pattern to be similar, I guess that, when new high-priority
I/O arrives, this I/O should preempt lower-priority I/O.
Unfortunately, this is not always the case, depending on other
parameters. Waker/wakee I/O is guaranteed to be injected only when the
in-service queue has no I/O.
At any rate, probably we can solve this puzzle by just analyzing a
trace in which you detect a loss of throughput without 7cc4ffc55564.
The best case would be one with the minimum possible number of
threads, to get a simpler trace.
> I just wanted to bounce
> this off of you if you have any suggestion what to look for or any tips
> regarding why 7cc4ffc55564 apparently achieves much more reliable request
> injection for dbench.
I hope my considerations above help a little bit.
Thanks,
Paolo
> Honza
> --
> Jan Kara <jack@xxxxxxxx>
> SUSE Labs, CR
