On 5/25/21 3:01 AM, Zhang, Qiang wrote: [...] >> Haven't looked at the trace and description, but I do think >> there is a problem it solves. >> >> 1) io_wait_on_hash() -> __add_wait_queue(&hash->wait, &wqe->wait); >> 2) (note: wqe is a worker) wqe's workers exit dropping refs >> 3) refs are zero, free io-wq >> 4) @hash is shared, so other task/wq does wake_up(&wq->hash->wait); >> 5) it wakes freed wqe >> >> step 4) is a bit more trickier than that, tl;dr; >> wq3:worker1 | locks bit1 >> wq1:worker2 | waits bit1 >> wq2:worker1 | waits bit1 >> wq1:worker3 | waits bit1 >> >> wq3:worker1 | drop bit1 >> wq1:worker2 | locks bit1 >> wq1:worker2 | completes all wq1 bit1 work items >> wq1:worker2 | drop bit1, exit and free io-wq >> >> wq2:worker1 | locks bit1 >> wq1 | free complete >> wq2:worker1 | drops bit1 >> wq1:worker3 | waked up, even though freed >> >> Can be simplified, don't want to waste time on that > > Thanks Pavel > > Your description is better. I have another question: under what circumstances will three io-wq(wq1, wq2, wq3) be created to share this @hash? Oops, missed the email. It's created by io_uring, and passed to io-wq, which is per-task and created on demand by io_uring. Can be achieved by a snippet just below, where threads haven't had io_uring instances before. thread1: ring = create_io_uring(); thread2: submit_sqes(ring); thread3: submit_sqes(ring); > > This kind of problem also occurs between two io-wq(wq1, wq2). Is the following description OK？ Yep, and I feel like there are cases simpler (and more likely) than the one I described. > > wq1:worker2 | locks bit1 > wq2:worker1 | waits bit1 > wq1:worker3 | waits bit1 > > wq1:worker2 | completes all wq1 bit1 work items > wq1:worker2 | drop bit1, exit and free io-wq > > wq2:worker1 | locks bit1 > wq1 | free complete > wq2:worker1 | drops bit1 > wq1:worker3 | waked up, even though freed -- Pavel Begunkov
