On 2022-03-23, Gautam Thaker <ghthaker@xxxxxxxxx> wrote: >> > processor : 31 >> > vendor_id : GenuineIntel >> > cpu family : 6 >> > model : 63 >> > model name : Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz >> > stepping : 2 >> > microcode : 0x46 >> > cpu MHz : 1200.000 >> > cache size : 20480 KB >> >> Your CPU is running at 1.2GHz although it is capable of 2.4GHz. This >> looks like you have CPU frequency scaling activated. Investigate: >> >> grep -i cpu_freq /boot/config-5.15.28-rt35 > > node-0> grep -i cpu_freq /boot/config-5.15.28-rt35 > CONFIG_ACPI_CPU_FREQ_PSS=y > CONFIG_CPU_FREQ=y > CONFIG_CPU_FREQ_GOV_ATTR_SET=y > CONFIG_CPU_FREQ_GOV_COMMON=y > CONFIG_CPU_FREQ_STAT=y > CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y > # CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set > # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set > # CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL is not set > CONFIG_CPU_FREQ_GOV_PERFORMANCE=y > CONFIG_CPU_FREQ_GOV_POWERSAVE=y > CONFIG_CPU_FREQ_GOV_USERSPACE=y > CONFIG_CPU_FREQ_GOV_ONDEMAND=y > CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y > CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y Your system is configured to support many different governors. For minimal latency it is important that you are using the performance governor. The frequency scaling might be the cause of the horrible hwlatdetect numbers you are seeing. If you have the cpufrequtils package installed, you can easily switch to the performance governor with: cpufreq-set -g performance With cpufreq-info you can see a nice summary of how your CPUs are currently set. You should see it running full speed. You can also configure the performance governor manually using sysfs. You may want to read through the documentation [0] so that you understand what you are doing. [0] https://www.kernel.org/doc/html/latest/admin-guide/pm/cpufreq.html >> The configuration of your tick may also be interesting: >> >> grep -i hz /boot/config-5.15.28-rt35 >> > > node-0> grep -i hz /boot/config-5.15.28-rt35 > CONFIG_NO_HZ_COMMON=y > # CONFIG_HZ_PERIODIC is not set > CONFIG_NO_HZ_IDLE=y > # CONFIG_NO_HZ_FULL is not set > CONFIG_NO_HZ=y > # CONFIG_HZ_100 is not set > CONFIG_HZ_250=y > # CONFIG_HZ_300 is not set > # CONFIG_HZ_1000 is not set > CONFIG_HZ=250 > # CONFIG_MACHZ_WDT is not set The tick configuration won't be responsible for the huge latencies you are seeing. But when you start getting your latency down, you may want to consider using CONFIG_HZ_PERIODIC with CONFIG_HZ_100 (or CONFIG_HZ_1000 if this is a machine with a huge network load). But these settings can be very dependent on the types of work your system is doing, so you will need to test the latency affects. Possibly you will notice no affects. Adding --secaligned to your cyclictest command can help to force situations where your latency will be worsened because of the tick. Optimally you want cyclictest to hit the worst case scenario. John Ogness
