On Tue, Aug 18, 2020 at 10:44 PM Greg KH <greg@xxxxxxxxx> wrote: > > On Tue, Aug 18, 2020 at 10:24:13PM +0530, Muni Sekhar wrote: > > On Tue, Aug 18, 2020 at 8:06 PM Greg KH <greg@xxxxxxxxx> wrote: > > > > > > On Tue, Aug 18, 2020 at 08:00:11PM +0530, Muni Sekhar wrote: > > > > Hi all, > > > > > > > > I’ve two identical Linux systems with only kernel differences. > > > > > > What are the differences in the kernels? > > You didn't answer this question, is this the same kernel source being > compared here? Same version? Same compiler? Everything identical? Both systems are having exactly the same hardware configuration. Compiler and kernel versions are different. One system has Ubuntu 16.04.4 LTS(4.4.0-66-generic kernel with gcc version 5.4.0) kernel and the other one has Ubuntu 18.04.4 LTS(4.15.0-91-generic kernel with gcc version 7.5.0). > > > > > While doing kernel profiling with perf, I got the below mentioned > > > > metrics for Scheduler benchmarks. > > > > > > > > 1st system (older kernel version compared to the other system) benchmark result: > > > > > > > > $ perf bench sched messaging -g 64 > > > > # Running 'sched/messaging' benchmark: > > > > # 20 sender and receiver processes per group > > > > # 64 groups == 2560 processes run > > > > > > > > Total time: 2.936 [sec] > > > > > > > > > > > > 2nd system benchmark result: > > > > > > > > $ perf bench sched messaging -g 64 > > > > # Running 'sched/messaging' benchmark: > > > > # 20 sender and receiver processes per group > > > > # 64 groups == 2560 processes run > > > > > > > > Total time: 10.074 [sec] > > > > > > > > > > > > So as per scheduler benchmark results, clearly a huge difference > > > > between two systems. > > > > Can anyone suggest to me how to dive deeper to know the root cause for > > > > it. > > > > > > Look a the differences between your different kernels, that would be a > > > great start :) > > I created the difference between two kernel config files and then > > tried to spot the CONFIG*SCHED* differences. > > Interestingly I see the difference in I/O scheduler config, 1st system > > is set to “deadline” and other one is set to “cfq”. So I made it equal > > by echoing to “/sys/block/<disk device>/queue/scheduler" but still no > > change in scheduler benchmark metrics. > > > > Is it the correct way to find the differences between kernels? If so, > > what other important CONFIG_* variables need to consider? > > > > > > $ cat config.patch | grep -i sched > > > > CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y > > CONFIG_CGROUP_SCHED=y > > CONFIG_FAIR_GROUP_SCHED=y > > # CONFIG_RT_GROUP_SCHED is not set > > # IO Schedulers > > @@ -369,10 +434,14 @@ CONFIG_IOSCHED_NOOP=y > > CONFIG_IOSCHED_DEADLINE=y > > CONFIG_IOSCHED_CFQ=y > > CONFIG_CFQ_GROUP_IOSCHED=y > > -CONFIG_DEFAULT_IOSCHED="deadline" > > +CONFIG_DEFAULT_IOSCHED="cfq" > > +CONFIG_MQ_IOSCHED_DEADLINE=m > > +CONFIG_MQ_IOSCHED_KYBER=m > > +CONFIG_IOSCHED_BFQ=m > > +CONFIG_BFQ_GROUP_IOSCHED=y > > CONFIG_SCHED_SMT=y > > CONFIG_SCHED_MC=y > > +CONFIG_SCHED_MC_PRIO=y > > +# CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL is not set > > +CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y > > There's lots of other options that affect performance, depending on your > specific benchmark, other than these. > > good luck! > > greg k-h -- Thanks, Sekhar _______________________________________________ Kernelnewbies mailing list Kernelnewbies@xxxxxxxxxxxxxxxxx https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
