On 2/22/21 5:20 PM, Ahmed S. Darwish wrote:
On Mon, Feb 22, 2021 at 04:28:30PM +0100, Jonathan Schwender wrote:
Hi John,
This patch adds the option --mainaffinity to specify the affinity of
the main pid.
This is mainly useful if you want to bind the main thread to a
different (e.g. housekeeping ) CPU than the measurement threads.
Do you also have any numbers showing different results with/without
"--mainaffinity"?
Sorry for the delay. I do have some numbers now, and there is a benefit
to using this option if CPU isolation + CAT are used. Otherwise it's not
really visible.
Rendered Markdown:
https://gist.github.com/jschwe/d4c46026aec57b10a2b0e6f72258b96e
# Testing proposed cyclictest --mainaffinity option
System information:
- 2 socket Intel Xeon E5-2643 v4 @ 3.40Ghz
- Turbo boost is disabled.
- Fedora 33 with kernel 5.10.1-rt20 with small patch (see cmdline isolcpus)
- Cmdline: `nosmt isolcpus=domain,managed_irq,wq,rcu,misc,kthread,3,5,7,9,11 rcu_nocbs=3,5,7,9,11 irqaffinity=0,2,4 maxcpus=12 rcu_nocb_poll nowatchdog tsc=nowatchdog processor.max_cstate=1 intel_idle.max_cstate=0 systemd.unified_cgroup_hierarchy=0`
- The additional isolcpus arguments set the HK_FLAG with the corresponding name.
This cmdline adds all HK_FLAGs usually set by nohz_full, except the actual
nohz flags `tick` and `timer`. This improves cyclictest latencies on my system.
- Rteval is running on all CPUs from node 0 + CPU 1, but not on the isolated CPUs.
- L3 Cache is reserved for the isolated CPUs via `resctrl` (CPU based allocation)
- Test duration 24 hours, interval 200 µs
## Test 1: 5 cyclictest instance with main pid on same cpu as the measurement thread
This test simply starts 5 cyclictest instances (via numactl) with one measurement thread each and bound to a single
CPU via `--affinity`, so that the main thread is also bound to the same CPU.
## Test 2: Single cyclictest instance with --mainaffinity=1 for isolated CPUs 3,5,7,9,11
The main thread was placed on CPU 1 via `--mainaffinity` and `--refresh_on_max` was added for good measure to keep the
logfile small.
Thanks,
--
Ahmed S. Darwish
Jonathan
