Hi Wido, Are you trying this setting? /sys/devices/system/cpu/intel_pstate/min_perf_pct -----Original Message----- From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of Wido den Hollander Sent: 14 May 2018 14:14 To: nick@xxxxxxxxxx; 'Blair Bethwaite' <blair.bethwaite@xxxxxxxxx> Cc: 'ceph-users' <ceph-users@xxxxxxxxxxxxxx> Subject: Re: Intel Xeon Scalable and CPU frequency scaling on NVMe/SSD Ceph OSDs On 05/01/2018 10:19 PM, Nick Fisk wrote: > 4.16 required? > https://www.phoronix.com/scan.php?page=news_item&px=Skylake-X-P-State- > Linux- > 4.16 > I've been trying with the 4.16 kernel for the last few days, but still, it's not working. The CPU's keep clocking down to 800Mhz I've set scaling_min_freq=scaling_max_freq in /sys, but that doesn't change a thing. The CPUs keep scaling down. Still not close to the 1ms latency with these CPUs :( Wido > > -----Original Message----- > From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of > Blair Bethwaite > Sent: 01 May 2018 16:46 > To: Wido den Hollander <wido@xxxxxxxx> > Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>; Nick Fisk > <nick@xxxxxxxxxx> > Subject: Re: Intel Xeon Scalable and CPU frequency > scaling on NVMe/SSD Ceph OSDs > > Also curious about this over here. We've got a rack's worth of R740XDs > with Xeon 4114's running RHEL 7.4 and intel-pstate isn't even active > on them, though I don't believe they are any different at the OS level > to our Broadwell nodes (where it is loaded). > > Have you tried poking the kernel's pmqos interface for your use-case? > > On 2 May 2018 at 01:07, Wido den Hollander <wido@xxxxxxxx> wrote: >> Hi, >> >> I've been trying to get the lowest latency possible out of the new >> Xeon Scalable CPUs and so far I got down to 1.3ms with the help of Nick. >> >> However, I can't seem to pin the CPUs to always run at their maximum >> frequency. >> >> If I disable power saving in the BIOS they stay at 2.1Ghz (Silver >> 4110), but that disables the boost. >> >> With the Power Saving enabled in the BIOS and when giving the OS all >> control for some reason the CPUs keep scaling down. >> >> $ echo 100 > /sys/devices/system/cpu/intel_pstate/min_perf_pct >> >> cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report >> errors and bugs to cpufreq@xxxxxxxxxxxxxxx, please. >> analyzing CPU 0: >> driver: intel_pstate >> CPUs which run at the same hardware frequency: 0 >> CPUs which need to have their frequency coordinated by software: 0 >> maximum transition latency: 0.97 ms. >> hardware limits: 800 MHz - 3.00 GHz >> available cpufreq governors: performance, powersave >> current policy: frequency should be within 800 MHz and 3.00 GHz. >> The governor "performance" may decide which speed to use >> within this range. >> current CPU frequency is 800 MHz. >> >> I do see the CPUs scale up to 2.1Ghz, but they quickly scale down >> again to 800Mhz and that hurts latency. (50% difference!) >> >> With the CPUs scaling down to 800Mhz my latency jumps from 1.3ms to >> 2.4ms on avg. With turbo enabled I hope to get down to 1.1~1.2ms on avg. >> >> $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor >> performance >> >> Everything seems to be OK and I would expect the CPUs to stay at >> 2.10Ghz, but they aren't. >> >> C-States are also pinned to 0 as a boot parameter for the kernel: >> >> processor.max_cstate=1 intel_idle.max_cstate=0 >> >> Running Ubuntu 16.04.4 with the 4.13 kernel from the HWE from Ubuntu. >> >> Has anybody tried this yet with the recent Intel Xeon Scalable CPUs? >> >> Thanks, >> >> Wido >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > -- > Cheers, > ~Blairo > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |