On Tue, 14 Oct 2014, Andreas Bluemle wrote: > Hi, > > > On Wed, 8 Oct 2014 16:55:38 -0700 > Paul Von-Stamwitz <PVonStamwitz@xxxxxxxxxxxxxx> wrote: > > > > > > > Hi, > > > > > > > > as mentioned during today's meeting, here are the kernel boot > > > > parameters > > > which I found to provide the basis for good performance results: > > > > > > > > processor.max_cstate=0 > > > > intel_idle.max_cstate=0 > > > > > > > > I understand these to basically turn off any power saving modes > > > > of the > > > CPU; the CPU's we are using are like > > > > Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz > > > > Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz > > > > > > > > At the BIOS level, we > > > > - turn off Hyperthraeding > > > > - turn off Turbo mode (in order ot not leave the specifications) > > > > - turn on frequency floor override > > > > > > > > We also assert that > > > > /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor > > > > is set to "performance" > > > > > > > > Using above we see a constant frequency at the maximum level > > > > allowed by > > > the CPU (except Turbo mode). > > > > > > How much performance do we gain by this? Till now i thought it's > > > just 1-3% so i'm still running ondemand govenor plus power savings. > > > > As always, it depends. I saw noticeable increases in some throughput > > tests (though I can't recall the % gain.) More important to me was > > that it made my fio results much more consistent. As we measure > > improvements, these settings remove some of the "system noise". > > > > Best, > > Paul > > > > There were two different aspects which showed improvemnt: > - code was executed faster > - thread switching delays were reduced significantly > > See the attached grahics. They show processing of a 4 kB write > request: processing at the Pipe::Reader is roughly 200 us in both > pictures, and sth. like 20 us at the OSD::Dispatcher. So there > is not much of a benefit here. > > But the delay between the end of the Pipe::Reader and the start > of the OSD::Dispatcher threads reduced really significantly. This test had a single outstanding IO, right? The question for me is if this reflect latencies we'd see under a realistic workload, where the are more IOs in flight and the CPUs aren't likely to be in low power states. I'm not sure how low the load needs to be before those states kick in and these latencies start to appear... sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
