On Tue, 2018-04-10 at 16:41 +0800, Ming Lei wrote: > On Mon, Apr 09, 2018 at 06:34:55PM -0700, Bart Van Assche wrote: > > If a completion occurs after blk_mq_rq_timed_out() has reset > > rq->aborted_gstate and the request is again in flight when the timeout > > Given rq->aborted_gstate is reset only for BLK_EH_RESET_TIMER, I > think you are addressing the race between normal completion and > the timeout of BLK_EH_RESET_TIMER. Yes, that's correct. I will make this more explicit. > > expires then a request will be completed twice: a first time by the > > timeout handler and a second time when the regular completion occurs. > > This patch looks simpler, and more like the previous model of > using blk_mark_rq_complete(). That's also correct. > I have one question: > > - with this patch, rq's state is updated atomically as cmpxchg. Suppose > one rq is timed out, the rq's state is updated as MQ_RQ_COMPLETE by > blk_mq_check_expired(), then ops->timeout() is run and > BLK_EH_RESET_TIMER is returned, so blk_mq_add_timer(req, MQ_RQ_COMPLETE, > MQ_RQ_IN_FLIGHT) is called to update rq's state into MQ_RQ_IN_FLIGHT. > > Now the original normal completion still may occur after rq's state > becomes MQ_RQ_IN_FLIGHT, and seems there is still the double completion > with this patch? Maybe I am wrong, please explain a bit. That scenario won't result in a double completion. After the timer has been reset the block driver blk_mq_complete_request() call will change the request state from MQ_RQ_IN_FLIGHT into MQ_RQ_COMPLETE. The next time blk_mq_check_expired() is called it will execute the following code: blk_mq_change_rq_state(rq, MQ_RQ_IN_FLIGHT, MQ_RQ_COMPLETE); That function call only changes the request state if the current state is IN_FLIGHT. However, the blk_mq_complete_request() call changed the request state into COMPLETE. Hence, the above blk_mq_change_rq_state() call will return false and the blk-mq timeout code will skip this request. If the request would already have been reused and would have been marked again as IN_FLIGHT then its deadline will also have been updated and the request will be skipped by the timeout code because its deadline has not yet expired. > And synchronize_rcu() is needed by Tejun's approach between marking > COMPLETE and handling this rq's timeout, and the time can be quite long, > I guess it might be easier to trigger? I have done what I could to trigger races between the regular completion path and the timeout code in my tests. Without this patch if I run the srp-test software I see crashes being reported in the rdma_rxe driver but with this patch applied I don't see any crashes with my tests. Bart.