On Tue, Sep 22, 2020 at 12:43:37PM +0800, JeffleXu wrote: > Thanks for replying. Comments embedded below. > > > On 9/13/20 10:00 PM, Ming Lei wrote: > > On Fri, Sep 11, 2020 at 07:40:14PM +0800, JeffleXu wrote: > > > Thanks for replying ;) > > > > > > > > > On 9/11/20 7:01 PM, Ming Lei wrote: > > > > On Fri, Sep 11, 2020 at 11:29:58AM +0800, Jeffle Xu wrote: > > > > > Splitted bios of one source bio can be enqueued into different CPU since > > > > > the submit_bio() routine can be preempted or fall asleep. However this > > > > > behaviour can't work well with iopolling. > > > > Do you have user visible problem wrt. io polling? If yes, can you > > > > provide more details? > > > No, there's no practical example yet. It's only a hint from the code base. > > > > > > > > > > > Currently block iopolling only polls the hardwar queue of the input bio. > > > > > If one bio is splitted to several bios, one (bio 1) of which is enqueued > > > > > into CPU A, while the others enqueued into CPU B, then the polling of bio 1 > > > > > will cotinuously poll the hardware queue of CPU A, though the other > > > > > splitted bios may be in other hardware queues. > > > > If it is guaranteed that the returned cookie is from bio 1, poll is > > > > supposed to work as expected, since bio 1 is the chained head of these > > > > bios, and the whole fs bio can be thought as done when bio1 .end_bio > > > > is called. > > > Yes, it is, thanks for your explanation. But except for polling if the input > > > bio has completed, one of the > > > > > > important work of polling logic is to reap the completion queue. Let's say > > > one bio is split into > > > > > > two bios, bio 1 and bio 2, both of which are enqueued into the same hardware > > > queue.When polling bio1, > > > > > > though we have no idea about bio2 at all, the polling logic itself is still > > > reaping the completion queue of > > > > > > this hardware queue repeatedly, in which case the polling logic still > > > stimulates reaping bio2. > > > > > > > > > Then what if these two split bios enqueued into two different hardware > > > queue? Let's say bio1 is enqueued > > > > > > into hardware queue A, while bio2 is enqueued into hardware queue B. When > > > polling bio1, though the polling > > > > > > logic is repeatedly reaping the completion queue of hardware queue A, it > > > doesn't help reap bio2. bio2 is reaped > > > > > > by IRQ as usual. This certainly works currently, but this behavior may > > > deviate the polling design? I'm not sure. > > > > > > > > > In other words, if we can ensure that all split bios are enqueued into the > > > same hardware queue, then the polling > > > > > > logic *may* be faster. > > __submit_bio_noacct_mq() returns cookie from the last bio in current->bio_list, and > > this bio should be the bio passed to __submit_bio_noacct_mq() when bio splitting happens. > > > > Suppose CPU migration happens during bio splitting, the last bio should be > > submitted to LLD much late than other bios, so when blk_poll() finds > > completion on the hw queue of the last bio, usually other bios should > > be completed already most of times. > > > > Also CPU migration itself causes much bigger latency, so it is reasonable to > > not expect good IO performance when CPU migration is involved. And CPU migration > > on IO task shouldn't have been done frequently. That said it should be > > fine to miss the poll in this situation. > > Yes you're right. After diving into the code of nvme driver, currently nvme > driver indeed allocate interrupt for polling queues, No, nvme driver doesn't allocate interrupt for poll queues, please see nvme_setup_irqs(). > > that is, reusing the interrupt used by admin queue. > > Jens had ever said that the interrupt may be disabled for queues working in > polling mode someday (from my colleague). If > > that is true, then this may become an issue. But at least now this indeed > works. What is the issue? Thanks, Ming
