On Thu, Jan 27, 2022 at 01:19:43PM +0100, Peter Zijlstra wrote:
> On Mon, Jan 24, 2022 at 10:46:17PM +0800, Tao Zhou wrote:
> > Hi Peter,
> >
> > On Thu, Jan 20, 2022 at 04:55:22PM +0100, Peter Zijlstra wrote:
> >
> > [...]
> >
> > > +/* pre-schedule() */
> > > +void umcg_wq_worker_sleeping(struct task_struct *tsk)
> > > +{
> > > + struct umcg_task __user *self = READ_ONCE(tsk->umcg_task);
> > > + int ret;
> > > +
> > > + if (!tsk->umcg_server) {
> > > + /*
> > > + * Already blocked before, the pages are unpinned.
> > > + */
> > > + return;
> > > + }
> > > +
> > > + /* Must not fault, mmap_sem might be held. */
> > > + pagefault_disable();
> > > +
> > > + ret = umcg_update_state(tsk, self, UMCG_TASK_RUNNING, UMCG_TASK_BLOCKED);
> > > + if (ret == -EAGAIN) {
> > > + /*
> > > + * Consider:
> > > + *
> > > + * self->state = UMCG_TASK_RUNNABLE | UMCG_TF_COND_WAIT;
> > > + * ...
> > > + * sys_umcg_wait();
> > > + *
> > > + * and the '...' code doing a blocking syscall/fault. This
> > > + * ensures that returns with UMCG_TASK_RUNNING, which will make
> >
> > /UMCG_TASK_RUNNING/UMCG_TASK_RUNNABLE/
>
> So the issue is that:
>
> self->state = UMCG_TASK_RUNNABLE | UMCG_TF_COND_WAIT;
>
> <#PF>
> umcg_sys_enter()
> umcg_pin_user_page()
> schedule()
> sched_submit_work()
> umcg_wq_worker_sleeping()
> umcg_update_state(tsk, self, UMCG_TASK_RUNNING, UMCG_TASK_BLOCKED) // -EAGAIN
> UMCG_DIE()
>
> Which is clearly not desirable.
>
> So this additinoal thing ensures that:
>
> umcg_update_state(tsk, self, UMCG_TASK_RUNNABLE, UMCG_TASK_BLOCKED) // 0
>
> umcg_sys_exit()
> umcg_update_state(tsk, self, UMCG_TASK_BLOCKED, UMCG_TASK_RUNNABLE);
> umcg_enqueue_and_wake()
>
> umcg_notify_resume()
> umcg_wait()
>
> // must be UMCG_TASK_RUNNING here
> </#PF>
>
> So when the pagefault finally does return, it will have:
> UMCG_TASK_RUNNING.
>
> Which will then make sys_umcg_wait() return -EAGAIN and around we go.
Thank you, Peter.
> > > + * sys_umcg_wait() return with -EAGAIN.
> > > + */
> > > + ret = umcg_update_state(tsk, self, UMCG_TASK_RUNNABLE, UMCG_TASK_BLOCKED);
> > > + }
> > > + if (ret)
> > > + UMCG_DIE_PF("state");
> > > +
> > > + if (umcg_wake_server(tsk))
> > > + UMCG_DIE_PF("wake");
> > > +
> > > + pagefault_enable();
> > > +
> > > + /*
> > > + * We're going to sleep, make sure to unpin the pages, this ensures
> > > + * the pins are temporary. Also see umcg_sys_exit().
> > > + */
> > > + umcg_unpin_pages();
> > > +}
