On Sat, Mar 23, 2024 at 6:12 PM Tejun Heo <tj@xxxxxxxxxx> wrote: > > Hello, Joel. > > On Fri, Mar 22, 2024 at 10:37:32PM -0400, Joel Fernandes wrote: > ... > > I was wondering about the comment above related to 'wakeup_preempt', could > > you clarify why it is not useful (NOOP) in the sched-ext class? > > > > wakeup_preempt() may be called via: > > sched_ttwu_pending() -> > > ttwu_do_activate() -> > > wakeup_preempt() > > > > > > at which point the enqueue of the task could have already happened via: > > > > sched_ttwu_pending() -> > > ttwu_do_activate() -> > > activate_task() -> > > enqueue_task() > > > > But the comment above says "task isn't tied to the CPU" ? > > In sched_ext, a scheduling queue isn't tied to a particular CPU. For > example, it's trivial to share a single global scheduling queue across the > whole system or any subset of CPUs. To support this behavior, tasks can be > hot-migrated in the dispatch path just before it starts executing: > > https://github.com/sched-ext/sched_ext/blob/sched_ext/kernel/sched/ext.c#L1335 > > So, the CPU picked by ops.select_cpu() in the enqueue path often doesn't > determine the CPU the task is going to execute on. If the picked CPU matches > the CPU the task is eventually going to run on, there's a small performance > advantage as the later hot migration can be avoided. > > As the task isn't actually tied to the CPU being picked, it's a bit awkward > to ask "does this task preempt this CPU?" Instead, preemption is implemented > by calling scx_bpf_kick_cpu() w/ SCX_KICK_PREEMPT or using the > SCX_ENQ_PREEMPT flag from the enqueue path which allows preempting any CPU. > Got it. I took some time to look at it some more. Now I am wondering why check_preempt_curr() has to be separately implemented for a class and why the enqueue() handler of each class cannot take care of preempting curr via setting resched flags. The only reason I can see is that, activate_task() is not always followed by a check_preempt_curr() and sometimes there is an unconditional resched_curr() happening following the call to activate_task(). But such issues don't affect sched_ext in its current form I guess. Btw, if sched_ext were to be implemented as a higher priority class above CFS [1], then check_preempt_curr() may preempt without even calling the class's check_preempt_curr() : void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags) { if (p->sched_class == rq->curr->sched_class) rq->curr->sched_class->check_preempt_curr(rq, p, flags); else if (sched_class_above(p->sched_class, rq->curr->sched_class)) resched_curr(rq); But if I understand, sched_ext is below CFS at the moment, so that should not be an issue. [1] By the way, now that I brought up the higher priority class thing, I might as well discuss it here :-D : One of my use cases is about scheduling high priority latency sensitive threads: I think if sched_ext could have 2 classes, one lower than CFS and one above CFS, that would be beneficial to those who want a gradual transition to use scx, instead of switching all tasks to scx at once. One reason is EAS (in CFS). It may be beneficial for people to use the existing EAS for everything but latency critical tasks (much like how people use RT class for those). This is quite involved and reimplementing EAS in BPF may be quite a project. Not that it shouldn't be implemented that way, but EAS is about a decade old with all kinds of energy modeling, math and what not. Having scx higher than cfs alongside the lower one is less of an invasive approach than switching everything on the system to scx. Do you have any opinions on that? If it makes sense, I can work on such an implementation. Another reason for this is, general purpose systems run very varied workloads, and big dramatic changes are likely to be reverted due to power and performance regressions. Hence, the request for a higher scx, so that we (high priority task scx users) can take baby steps. thanks, - Joel
