Hi Jeremy, I think the reason why we observe the performance degradation after applying wastedcores patches is this issue: https://github.com/jplozi/wastedcores/issues/6 With HT turned OFF we see that wastedcores kernel compares as following to mainline kernel - performance regression in order of 1-5% for SPECjbb2005 and for Linpack and Stream - SPECjvm shows some minor improvements for some subtests but also some regressions for other subtests I think we will need to test the four proposed "wastedcores" patches one by one. Could you please prepare 4 different kernels where always just one patch from https://github.com/Freeaqingme/wastedcores will be applied? Jirka On Tue, May 3, 2016 at 11:43 PM, Jirka Hladky <jhladky@xxxxxxxxxx> wrote: > Hi Jeremy, > > please have a look at Linpack and Stream results > > > http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData.tar.xz > > and > > > http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData.tar.xz > > We have collected /proc/sched_debug every 10 seconds - there are > ""*scheddebug" files for every run. We are also collecting mpstat > statistics every 2 seconds. > > For example during a test when 64 stream jobs were running in parallel > there are corresponding /proc/sched_debug logs: > > > 4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug > <http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug> > > > 4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug > <http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug> > > However, we are not sure how to process these log files. Is there any tool > to do so or could you share some hints what to look for? > > Thanks a lot! > Jirka > > On Tue, May 3, 2016 at 4:18 PM, Jeremy Eder <jeder@xxxxxxxxxx> wrote: > >> Yikes. 20% off...why? Do you have any system stats? Particularly >> /proc/sched_debug collected at some interval. >> >> On Mon, May 2, 2016 at 12:39 PM, Jirka Hladky <jhladky@xxxxxxxxxx> wrote: >> >>> Hi all, >>> >>> we have compared scheduler performance >>> for kernel-4.6.0-0.rc5.git0.1.wastedcores >>> against kernel-4.6.0-0.rc5.git0.1.el7. >>> >>> *Summary: *against all expectations* kernel-4.6.0-0.rc5.git0.1.wastedcores performs on >>> average 20% WORSE than the regular kernel. * >>> >>> Results for different servers: >>> >>> SPECjbb2005: >>> >>> - karkulka-02.lab.eng.brq.redhat.com-htON >>> <http://perf-desktop.brq.redhat.com/SPECjbb/.vs./karkulka-02.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64-throughput-performancevs4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64-throughput-performance/Result-2016-Apr-26_03h05m44sResult-2016-Apr-29_22h17m07s/specjbb2005.html> >>> - kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON >>> <http://perf-desktop.brq.redhat.com/SPECjbb/.vs./kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64-throughput-performancevs4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64-throughput-performance/Result-2016-Apr-26_01h47m36sResult-2016-Apr-30_02h21m38s/specjbb2005.html> >>> >>> SPECjvm2008: >>> >>> - karkulka-02.lab.eng.brq.redhat.com-htON >>> <http://perf-desktop.brq.redhat.com/SPECjvm/.vs./karkulka-02.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64-throughput-performancevs4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64-throughput-performance/Result-2016-Apr-26_07h58m09sResult-2016-Apr-30_03h09m23s/SPECjvm2008.html> >>> - kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON >>> <http://perf-desktop.brq.redhat.com/SPECjvm/.vs./kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64-throughput-performancevs4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64-throughput-performance/Result-2016-Apr-26_05h06m46sResult-2016-Apr-30_05h40m50s/SPECjvm2008.html> >>> >>> Linpack and Stream: >>> >>> - karkulka-02.lab.eng.brq.redhat.com-htON >>> <http://perf-desktop.brq.redhat.com/LinpackAndStream/.vs./karkulka-02.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64-throughput-performancevs4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64-throughput-performance/Results-2016-Apr-25_21h38m02sResults-2016-Apr-29_16h22m44s/linpackAndStreamResults.html> >>> - kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON >>> <http://perf-desktop.brq.redhat.com/LinpackAndStream/.vs./kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64-throughput-performancevs4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64-throughput-performance/Results-2016-Apr-25_21h36m46sResults-2016-Apr-29_21h37m03s/linpackAndStreamResults.html> >>> >>> Any comments, questions, suggestions? >>> >>> Jirka and Kamil >>> >>> On Thu, Apr 28, 2016 at 12:25 AM, Jirka Hladky <jhladky@xxxxxxxxxx> >>> wrote: >>> >>>> Hi Jeremy, >>>> >>>> I have just read >>>> >>>> http://www.i3s.unice.fr/~jplozi/wastedcores/files/extended_talk.pdf >>>> http://www.ece.ubc.ca/~sasha/papers/eurosys16-final29.pdf >>>> >>>> We will definitely test it and share the results here. >>>> >>>> Jirka >>>> >>>> >>>> >> >> >> -- >> >> -- Jeremy Eder >> > > _______________________________________________ kernel mailing list kernel@xxxxxxxxxxxxxxxxxxxxxxx http://lists.fedoraproject.org/admin/lists/kernel@xxxxxxxxxxxxxxxxxxxxxxx
