On Tue, Jan 18, 2022 at 10:19:21AM -0800, Peter Oskolkov wrote: > ============= worker-to-worker context switches > > One example: absl::Mutex (https://abseil.io/about/design/mutex) has > google-internal extensions that are "fiber aware". More specifically, > consider this situation: > > - worker W1 acqured the mutex and is doing its work > - worker W2 calls mutex::lock() > mutex::lock(), being aware of workers, understands that W2 is going to sleep; > so instead of just doing so, waking the server, and letting > the server figure out what to run in place of the sleeping worker, > mutex::lock() > calls into the userspace scheduler in the context of W2 running, and the > userspace scheduler then picks W3 to run and does W2->W3 context switch. > > The optimization above replaces W2->Server and Server->W3 context switches > with a single W2->W3 context switch, which is a material performance gain. Yes, I've also already reconsidered. Things like pipelines and other fixed order scheduling policies will greatly benefit from worker-to-worker switching. But I think all of them are explicit. That is, we can limit the ::next_tid usage to sys_umcg_wait() and never look at it for implicit blocks. > In addition, when W1 calls mutex::unlock(), the scheduling code determines > that W2 is waiting on the mutex, and thus calls W2::wake() from the context of > running W1 (you asked earlier why do we need "WAKE_ONLY"). This I'm not at all convinced on. That sounds like it will violate the 1:1 thing.
