On 05/14/2018 04:46 PM, Nick Fisk wrote: > Hi Wido, > > Are you trying this setting? > > /sys/devices/system/cpu/intel_pstate/min_perf_pct > Yes, but that doesn't help. I can set it to 80, 100 or any value I like, the CPUs keep clocking down to 800Mhz. At first I was having some issues with getting intel_pstate loaded, but with 4.16 it loaded without any problems, but still, CPUs keep clocking down. Wido > > > -----Original Message----- > From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of Wido den > Hollander > Sent: 14 May 2018 14:14 > To: nick@xxxxxxxxxx; 'Blair Bethwaite' <blair.bethwaite@xxxxxxxxx> > Cc: 'ceph-users' <ceph-users@xxxxxxxxxxxxxx> > Subject: Re: Intel Xeon Scalable and CPU frequency scaling on > NVMe/SSD Ceph OSDs > > > > On 05/01/2018 10:19 PM, Nick Fisk wrote: >> 4.16 required? >> https://www.phoronix.com/scan.php?page=news_item&px=Skylake-X-P-State- >> Linux- >> 4.16 >> > > I've been trying with the 4.16 kernel for the last few days, but still, it's > not working. > > The CPU's keep clocking down to 800Mhz > > I've set scaling_min_freq=scaling_max_freq in /sys, but that doesn't change > a thing. The CPUs keep scaling down. > > Still not close to the 1ms latency with these CPUs :( > > Wido > >> >> -----Original Message----- >> From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of >> Blair Bethwaite >> Sent: 01 May 2018 16:46 >> To: Wido den Hollander <wido@xxxxxxxx> >> Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>; Nick Fisk >> <nick@xxxxxxxxxx> >> Subject: Re: Intel Xeon Scalable and CPU frequency >> scaling on NVMe/SSD Ceph OSDs >> >> Also curious about this over here. We've got a rack's worth of R740XDs >> with Xeon 4114's running RHEL 7.4 and intel-pstate isn't even active >> on them, though I don't believe they are any different at the OS level >> to our Broadwell nodes (where it is loaded). >> >> Have you tried poking the kernel's pmqos interface for your use-case? >> >> On 2 May 2018 at 01:07, Wido den Hollander <wido@xxxxxxxx> wrote: >>> Hi, >>> >>> I've been trying to get the lowest latency possible out of the new >>> Xeon Scalable CPUs and so far I got down to 1.3ms with the help of Nick. >>> >>> However, I can't seem to pin the CPUs to always run at their maximum >>> frequency. >>> >>> If I disable power saving in the BIOS they stay at 2.1Ghz (Silver >>> 4110), but that disables the boost. >>> >>> With the Power Saving enabled in the BIOS and when giving the OS all >>> control for some reason the CPUs keep scaling down. >>> >>> $ echo 100 > /sys/devices/system/cpu/intel_pstate/min_perf_pct >>> >>> cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report >>> errors and bugs to cpufreq@xxxxxxxxxxxxxxx, please. >>> analyzing CPU 0: >>> driver: intel_pstate >>> CPUs which run at the same hardware frequency: 0 >>> CPUs which need to have their frequency coordinated by software: 0 >>> maximum transition latency: 0.97 ms. >>> hardware limits: 800 MHz - 3.00 GHz >>> available cpufreq governors: performance, powersave >>> current policy: frequency should be within 800 MHz and 3.00 GHz. >>> The governor "performance" may decide which speed to > use >>> within this range. >>> current CPU frequency is 800 MHz. >>> >>> I do see the CPUs scale up to 2.1Ghz, but they quickly scale down >>> again to 800Mhz and that hurts latency. (50% difference!) >>> >>> With the CPUs scaling down to 800Mhz my latency jumps from 1.3ms to >>> 2.4ms on avg. With turbo enabled I hope to get down to 1.1~1.2ms on avg. >>> >>> $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor >>> performance >>> >>> Everything seems to be OK and I would expect the CPUs to stay at >>> 2.10Ghz, but they aren't. >>> >>> C-States are also pinned to 0 as a boot parameter for the kernel: >>> >>> processor.max_cstate=1 intel_idle.max_cstate=0 >>> >>> Running Ubuntu 16.04.4 with the 4.13 kernel from the HWE from Ubuntu. >>> >>> Has anybody tried this yet with the recent Intel Xeon Scalable CPUs? >>> >>> Thanks, >>> >>> Wido >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> >> >> -- >> Cheers, >> ~Blairo >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |