> Il giorno 11 gen 2021, alle ore 14:15, yukuai (C) <yukuai3@xxxxxxxxxx> ha scritto: > > Hi, > > We found a performance problem: > > kernel version: 5.10 > disk: ssd > scheduler: bfq > arch: arm64 / x86_64 > test param: direct=1, ioengine=psync, bs=4k, rw=randread, numjobs=32 > > We are using 32 threads here, test results showed that iops is equal > to single thread. > > After digging into the problem, I found root cause of the problem is strange: > > bfq_add_request > bfq_bfqq_handle_idle_busy_switch > bfq_add_bfqq_busy > bfq_activate_bfq > bfq_activate_requeue_entity > __bfq_activate_requeue_entity > __bfq_activate_entity > if (!bfq_entity_to_bfqq(entity)) > if (!entity->in_groups_with_pending_reqs) > entity->in_groups_with_pending_reqs = true; > bfqd->num_groups_with_pending_reqs++ > > If test process is not in root cgroup, num_groups_with_pending_reqs will > be increased after request was instered to bfq. > > bfq_select_queue > bfq_better_to_idle > idling_needed_for_service_guarantees > bfq_asymmetric_scenario > return varied_queue_weights || multiple_classes_busy || bfqd->num_groups_with_pending_reqs > 0 > > After issuing IO to driver, num_groups_with_pending_reqs is ensured to > be nonzero, thus bfq won't expire the queue. This is the root cause of > degradating to single-process performance. > > One the other hand, if I set slice_idle to zero, bfq_better_to_idle will > return false early, and the problem will disapear. However, relative > control will be inactive. > > My question is that, is this a known flaw for bfq? If not, as cfq don't > have such problem, is there a suitable solution? > Hi, this is a core problem, not of BFQ but of any possible solution that has to provide bandwidth isolation with sync I/O. One of the examples is the one I made for you in my other email. At any rate, the problem that you report seems to occur with just one group. We may think of simply changing my condition bfqd->num_groups_with_pending_reqs > 0 to bfqd->num_groups_with_pending_reqs > 1 If this simple solution does solve the problem you report, then I could run my batch of tests to check whether it causes some regression. What do you think? Thanks. Paolo > Thanks! > Yu Kuai > such problem,
