On Wed, 5 Aug 2015, Ding Dinghua wrote: > 2015-08-05 0:13 GMT+08:00 Somnath Roy <Somnath.Roy@xxxxxxxxxxx>: > > Yes, it has to re-acquire pg_lock today.. > > But, between journal write and initiating the ondisk ack, there is one context switche in the code path. So, I guess the pg_lock is not the only one that is causing this 1 ms delay... > > Not sure increasing the finisher threads will help in the pg_lock case as it will be more or less serialized by this pg_lock.. > My concern is, if pg lock of pg A has been grabbed, not only ondisk > callback of pg A is delayed, since ondisk_finisher has only one > thread, ondisk callback of other pgs will be delayed too. I wonder if an optimistic approach might help here by making the completion synchronous and doing something like if (pg->lock.TryLock()) { pg->_finish_thing(completion->op); delete completion; } else { finisher.queue(completion); } or whatever. We'd need to ensure that we aren't holding any lock or throttle budget that the pg could deadlock against. sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html