> (Actually when I take a step back and read your motivational example > of a legacy game in VM, I don't think FIFO (or another RT policy) is > suited for this case. Plain SCHED_OTHER and cpu controller's > bandwidth limitation could do just fine -- you can apply to a > (threaded) cgroup with chosen threads only.) As you might have already surmised it was a placeholder example to give the general idea. I think it is time to add some more details. :) Assume that you have an embedded system running some kind of software with real time like properties. You want to develop and debug your software locally on your high-end machine since it is much more convenient. Alas the software runs way too fast due to the difference in performance so you can't detect overruns etc. Assume also that you have a special scheduler that behaves in a very strange way compared to ordinary schedulers. Each scheduling tick it waits a configurable time before it lets the scheduled task execute its tick. This effectively means that the task is throttled and thus executes slower than it normally would. By tuning the waiting time you can tune the execution speed so comes close enough to what it is on the target system. Assume also that you have enough cores in your system so you can dedicate one core for the VMM and one core for the virtual core thread the Guest uses to execute the software. One way of implementing this kind of scheduler would be to create a fork of the FIFO scheduler that have this behavior. This crazy(?) technique can of course be used for other things such as running old DOS games or running an RTOS intended for embedded systems as the Guest in the VM and so on. ;) /John -----Original Message----- From: Michal Koutný <mkoutny@xxxxxxxx> Sent: den 15 mars 2022 11:36 To: Olsson John <john.olsson@xxxxxxxxxxxxx> Cc: cgroups@xxxxxxxxxxxxxxx Subject: Re: [EXTERNAL] Re: Split process across multiple schedulers? On Tue, Mar 15, 2022 at 08:19:26AM +0000, Olsson John <john.olsson@xxxxxxxxxxxxx> wrote: > If I'm understanding you correctly this effectively means that it is > possible to spread a process and its threads across multiple cgroups > that in turn may have different schedulers (and CPU affinity) > associated with them? Yes, the docs is here https://www.kernel.org/doc/html/v5.17-rc8/admin-guide/cgroup-v2.html#threads > > (Without CONFIG_RT_GROUP_SCHED all RT threads are effectively in the > > root cgroup.) > > Interesting! I have missed this little tidbit of information. This is > indeed very good to know! Maybe I should have added this applies from the POV of the cpu controller in particular... > A side effect of this is that in V2 you can't have an RT thread pinned > to a specific core that is evacuated, right? ...it has no effect for grouping of cpuset controller (assuming both cpu and cpuset are enabled in given subtree). > If you could do this it would also be possible to remove the portion > of the scheduling interval that is left for non-RT threads in the > cgroup config since there would not be any other threads on this > evacuated core. > By doing that you would eliminate jitter due to that otherwise the > scheduler would interrupt the RT thread and immediately re-schedule it > again. And thus you would theoretically get very good RT properties > (unless you make system calls). Well, there are more jobs that can interfere with RT workload on a cpu (see isolcpus= [1]) and there's some ongoing work to make these more convenient via cpuset controller [2]. The currently working approach would be to use isolcpus= cmdline to isolate the CPUs and use either sched_setaffinity() or cpuset controller to place tasks on the reserved CPUs (the cpuset approach is more strict as it may prevent unprivileged threads to switch to another CPU). > If you instead used FIFO scheduling (which handles RT threads only, > right?) then you could eliminate this noise. Or I am just showing off > how little I understand about scheduling in Linux. ;) (Actually when I take a step back and read your motivational example of a legacy game in VM, I don't think FIFO (or another RT policy) is suited for this case. Plain SCHED_OTHER and cpu controller's bandwidth limitation could do just fine -- you can apply to a (threaded) cgroup with chosen threads only.) HTH, Michal [1] https://www.kernel.org/doc/html/v5.17-rc8/admin-guide/kernel-parameters.html?highlight=isolcpus [2] https://lore.kernel.org/all/20211205183220.818872-1-longman@xxxxxxxxxx/
