On 04/06/2019 07:36 AM, Keith Busch wrote: > On Fri, Apr 5, 2019 at 5:04 PM Jens Axboe <axboe@xxxxxxxxx> wrote: >> Looking at current peak testing, I've got around 1.2% in queue enter >> and exit. It's definitely not free, hence my question. Probably safe >> to assume that we'll double that cycle counter, per IO. > > Okay, that's not negligible at all. I don't know of a faster reference > than the percpu_ref, but that much overhead would have to rule out > having a per hctx counter. > If there is no faster reference to enable waiting for all inflight requests to complete, is it possible to re-map (migrate) those requests to other hctx whose cpus (ctx) are still online, e.g., to extract the bio and re-map those bio to other ctx (e.g., cpu0)? One drawback I can see is if 63 out of 64 cpus are suddenly offline, cpu0 would be stuck. Dongli Zhang