On 10/13/22 08:07, Christophe Blaess wrote:
> `sysctl kernel.sched_rt_runtime_us=-1`
rt throttling is probably the reason for the poor results. Mainly
because:
"and lots of idle time on the CPU"
The CPU goes idle because of rt throttling.
Anyways, running busy-loop tasks with real-time priority is just
wrong and misleading.
If your goal is to run a single task in a CPU, do cpu isolation.
Ideally, with CPU isolation and a single-user app, *no scheduler is
needed,* and real-time schedulers are not designed for this of
use-case.
In practice (assuming proper cpu isolation), some kernel-related
tasks might be placed in the isolated CPU. Postponing
their execution will put the system in a potentially faulty state.
To make progress one of the two "workarounds" are needed:
- the user application needs to do periodic sleep to yield
some CPU.
- to user needs to use things like stalld to boost the
starving task for a limited amount of time (like 20 us)
using SCHED_DEADLINE.
Either way, the results will likely be worse than running the
application as SCHED_OTHER. The reason is that the current
status of cpu isolation is so good that any type of mechanism
to avoid/deal with starvation will likely be more costly
then just letting the kernel do some work in the CPU here and there.
See this paper (it is in ieeexplorer but has open access -
Thanks, Sant'Anna).
- Operating System Noise in the Linux kernel
https://ieeexplore.ieee.org/document/9812514
It shows that, with the system tuned, the observed
single latency of FIFO and SCHED other are very similar.
Yes, FIFO shows an improvement of 2 us in the max single
latency, but:
- the FIFO is doing some micro sleeps to avoid starvation - and they
are higher than 10 us.
- The FIFO is receiving more overall noise. See the peak of noise
occurrences in lower latency numbers.
So in the end, the SCHED_OTHER is yielding more cpu time (less accumulative
noise), with a similar max single noise.
Thus, yes, for busy loop workload "SCHED_OTHER > SCHED_FIFO," because
SCHED_FIFO was not designed for this scenario. This scenario is a
no scheduler at all goal (even SCHED_IDLE should be > SCHED_FIFO, and
SCHED_FIFO > than SCHED_DEADLINE).
CPU Isolation is at a very good stage, and the correct fix for arbitrary
kernel workload being placed in isolated CPUs is to fix the code, not
"the scheduler."
-- Daniel
