Hello, we reported "[brauner-vfs:vfs.misc.backing_file] [fs/pipe] cc03a5d65a: stress-ng.pipeherd.ops_per_sec -22.9% regression" in https://lore.kernel.org/all/202310261611.90b9e38c-oliver.sang@xxxxxxxxx/ when this commit is commit: cc03a5d65a4032f8c53940927343c1795f2d2c53 ("fs/pipe: remove unnecessary spinlock from pipe_write()") https://git.kernel.org/cgit/linux/kernel/git/vfs/vfs.git vfs.misc.backing_file now we observed similar regression while this commit is merged in mainline. at the same time, we also observed improvement in unixbench tests. below details just FYI. kernel test robot noticed a -26.1% regression of stress-ng.pipeherd.ops_per_sec on: commit: dfaabf916b1ca83cfac856745db2fc9d57d9b13a ("fs/pipe: remove unnecessary spinlock from pipe_write()") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master testcase: stress-ng test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory parameters: nr_threads: 10% disk: 1HDD testtime: 60s fs: ext4 class: os test: pipeherd cpufreq_governor: performance In addition to that, the commit also has significant impact on the following tests: +------------------+-------------------------------------------------------------------------------------------------+ | testcase: change | unixbench: unixbench.throughput 51.8% improvement | | test machine | 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory | | test parameters | cpufreq_governor=performance | | | nr_task=100% | | | runtime=300s | | | test=pipe | +------------------+-------------------------------------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.pipeherd.ops_per_sec -34.2% regression | | test machine | 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 128G memory | | test parameters | class=memory | | | cpufreq_governor=performance | | | nr_threads=1 | | | test=pipeherd | | | testtime=60s | +------------------+-------------------------------------------------------------------------------------------------+ If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <oliver.sang@xxxxxxxxx> | Closes: https://lore.kernel.org/oe-lkp/202401181624.1534a30e-oliver.sang@xxxxxxxxx Details are as below: --------------------------------------------------------------------------------------------------> The kernel config and materials to reproduce are available at: https://download.01.org/0day-ci/archive/20240118/202401181624.1534a30e-oliver.sang@xxxxxxxxx ========================================================================================= class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime: os/gcc-12/performance/1HDD/ext4/x86_64-rhel-8.3/10%/debian-11.1-x86_64-20220510.cgz/lkp-icl-2sp7/pipeherd/stress-ng/60s commit: b4bd6b4bac ("fs/pipe: move check to pipe_has_watch_queue()") dfaabf916b ("fs/pipe: remove unnecessary spinlock from pipe_write()") b4bd6b4bac8edd61 dfaabf916b1ca83cfac856745db ---------------- --------------------------- %stddev %change %stddev \ | \ 7179 ± 20% -18.3% 5863 ± 8% uptime.idle 3.346e+09 ± 22% -28.0% 2.411e+09 ± 9% cpuidle..time 1.046e+08 -26.5% 76903274 cpuidle..usage 75.94 ± 4% -16.3% 63.57 ± 2% iostat.cpu.idle 22.46 ± 12% +55.4% 34.90 ± 4% iostat.cpu.system 75.35 ± 4% -12.8 62.55 ± 3% mpstat.cpu.all.idle% 21.81 ± 14% +13.0 34.78 ± 5% mpstat.cpu.all.sys% 58684 ± 14% -44.6% 32531 ± 68% numa-numastat.node0.other_node 1412129 ± 6% -35.2% 914436 ± 42% numa-numastat.node1.local_node 1419809 ± 6% -33.2% 948240 ± 40% numa-numastat.node1.numa_hit 75.96 ± 4% -16.3% 63.60 ± 2% vmstat.cpu.id 6865831 -12.4% 6015779 vmstat.memory.cache 15.39 ± 14% +47.7% 22.73 ± 5% vmstat.procs.r 1088887 ± 37% -68.8% 339310 ± 36% numa-meminfo.node1.Inactive 1080606 ± 38% -69.4% 330949 ± 37% numa-meminfo.node1.Inactive(anon) 915963 ± 35% -51.3% 446173 ± 26% numa-meminfo.node1.Mapped 3488008 ± 8% -40.9% 2061072 ± 51% numa-meminfo.node1.Shmem 6742528 -12.7% 5887524 meminfo.Cached 1363945 ± 31% -57.6% 577677 ± 7% meminfo.Inactive 1351521 ± 31% -58.2% 565253 ± 7% meminfo.Inactive(anon) 8298482 -10.4% 7431362 meminfo.Memused 3856294 ± 2% -20.9% 3049597 ± 2% meminfo.Shmem 8504922 -10.3% 7631133 meminfo.max_used_kB 1.036e+08 -26.1% 76601132 stress-ng.pipeherd.ops 1725189 -26.1% 1275768 stress-ng.pipeherd.ops_per_sec 1918 ± 7% +60.4% 3077 ± 7% stress-ng.time.involuntary_context_switches 1677 +37.1% 2300 stress-ng.time.percent_of_cpu_this_job_got 1015 +38.1% 1401 stress-ng.time.system_time 1.036e+08 -26.1% 76594217 stress-ng.time.voluntary_context_switches 58684 ± 14% -44.6% 32531 ± 68% numa-vmstat.node0.numa_other 270039 ± 38% -69.3% 82836 ± 37% numa-vmstat.node1.nr_inactive_anon 228757 ± 35% -51.2% 111712 ± 26% numa-vmstat.node1.nr_mapped 872073 ± 8% -41.0% 514493 ± 51% numa-vmstat.node1.nr_shmem 270039 ± 38% -69.3% 82836 ± 37% numa-vmstat.node1.nr_zone_inactive_anon 1419989 ± 6% -33.2% 948013 ± 40% numa-vmstat.node1.numa_hit 1412309 ± 6% -35.3% 914209 ± 42% numa-vmstat.node1.numa_local 1078 ± 15% +32.3% 1426 ± 5% turbostat.Avg_MHz 35.18 ± 14% +11.0 46.20 ± 4% turbostat.Busy% 680452 ± 3% -32.0% 462647 ± 2% turbostat.C1 1.029e+08 -26.1% 76043040 turbostat.C1E 946815 ± 81% -64.7% 334320 ± 71% turbostat.C6 17.72 ± 67% -9.7 8.05 ± 57% turbostat.C6% 64.50 ± 8% -17.2% 53.44 ± 4% turbostat.CPU%c1 0.11 -24.2% 0.08 ± 5% turbostat.IPC 7862367 ± 9% -8.8% 7167100 ± 3% turbostat.IRQ 63090 ± 4% -17.9% 51823 ± 2% turbostat.POLL 1687064 -12.7% 1473145 proc-vmstat.nr_file_pages 338030 ± 31% -58.2% 141419 ± 7% proc-vmstat.nr_inactive_anon 963697 -20.9% 761859 ± 2% proc-vmstat.nr_shmem 26816 -2.2% 26214 proc-vmstat.nr_slab_reclaimable 718428 -1.7% 706356 proc-vmstat.nr_unevictable 338030 ± 31% -58.2% 141419 ± 7% proc-vmstat.nr_zone_inactive_anon 718428 -1.7% 706356 proc-vmstat.nr_zone_unevictable 1749576 -13.8% 1508913 proc-vmstat.numa_hit 1683212 ± 2% -14.3% 1442579 proc-vmstat.numa_local 636656 ± 11% -63.8% 230268 ± 4% proc-vmstat.pgactivate 1802369 -13.4% 1560001 proc-vmstat.pgalloc_normal 636575 ± 4% -9.8% 574253 ± 3% proc-vmstat.pgfault 471340 ± 3% -6.4% 441076 proc-vmstat.pgfree 652416 ± 10% -11.6% 577024 ± 4% proc-vmstat.unevictable_pgs_scanned 0.00 ± 27% +70.4% 0.01 ± 17% perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.__mutex_lock.constprop.0.pipe_read 0.01 +16.7% 0.01 perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread 0.01 ± 23% +427.8% 0.03 ± 26% perf-sched.sch_delay.max.ms.schedule_preempt_disabled.__mutex_lock.constprop.0.pipe_read 0.70 ± 3% +22.8% 0.86 perf-sched.total_wait_and_delay.average.ms 5319883 ± 3% -19.4% 4286519 perf-sched.total_wait_and_delay.count.ms 0.69 ± 3% +22.8% 0.85 perf-sched.total_wait_time.average.ms 216.66 ± 6% -13.7% 186.90 ± 10% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.57 ± 3% +23.0% 0.70 perf-sched.wait_and_delay.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64 13.61 ± 5% +32.5% 18.03 ± 11% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread 5317357 ± 3% -19.4% 4283821 perf-sched.wait_and_delay.count.pipe_read.vfs_read.ksys_read.do_syscall_64 361.00 ± 5% -23.6% 275.83 ± 11% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread 0.55 ± 5% +21.7% 0.67 ± 2% perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock.pipe_read.vfs_read.ksys_read 0.45 ± 45% +49.3% 0.67 ± 5% perf-sched.wait_time.avg.ms.__cond_resched.pipe_read.vfs_read.ksys_read.do_syscall_64 216.65 ± 6% -13.7% 186.90 ± 10% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.54 ± 5% +23.4% 0.66 ± 2% perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64 0.57 ± 3% +23.1% 0.70 perf-sched.wait_time.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64 13.60 ± 5% +32.5% 18.02 ± 11% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread 0.46 ± 45% +66.5% 0.76 ± 15% perf-sched.wait_time.max.ms.__cond_resched.pipe_read.vfs_read.ksys_read.do_syscall_64 0.86 ± 3% +14.6% 0.98 ± 6% perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64 0.57 ± 17% +66.9% 0.95 ± 8% perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.constprop.0.pipe_read 163887 +76.6% 289408 sched_debug.cfs_rq:/.avg_vruntime.avg 290077 ± 12% +25.8% 364822 ± 5% sched_debug.cfs_rq:/.avg_vruntime.max 135347 ± 10% +101.6% 272926 ± 2% sched_debug.cfs_rq:/.avg_vruntime.min 167779 ± 2% +72.6% 289572 ± 2% sched_debug.cfs_rq:/.left_vruntime.max 40973 ± 14% +49.5% 61238 ± 11% sched_debug.cfs_rq:/.left_vruntime.stddev 163887 +76.6% 289408 sched_debug.cfs_rq:/.min_vruntime.avg 290077 ± 12% +25.8% 364822 ± 5% sched_debug.cfs_rq:/.min_vruntime.max 135347 ± 10% +101.6% 272926 ± 2% sched_debug.cfs_rq:/.min_vruntime.min 167779 ± 2% +72.6% 289572 ± 2% sched_debug.cfs_rq:/.right_vruntime.max 40973 ± 14% +49.5% 61238 ± 11% sched_debug.cfs_rq:/.right_vruntime.stddev 377.43 ± 3% +12.3% 424.02 ± 3% sched_debug.cfs_rq:/.runnable_avg.avg 104.83 ± 7% +28.2% 134.42 ± 10% sched_debug.cfs_rq:/.runnable_avg.min 375.36 ± 3% +12.2% 421.09 ± 3% sched_debug.cfs_rq:/.util_avg.avg 104.25 ± 7% +28.5% 134.00 ± 10% sched_debug.cfs_rq:/.util_avg.min 9255 ± 6% +13.1% 10472 ± 4% sched_debug.cpu.avg_idle.min 1275 ± 9% +25.0% 1594 ± 8% sched_debug.cpu.curr->pid.avg 0.28 ± 10% +26.3% 0.35 ± 7% sched_debug.cpu.nr_running.avg 1528268 -26.1% 1129961 sched_debug.cpu.nr_switches.avg 1646995 ± 2% -25.3% 1230736 sched_debug.cpu.nr_switches.max 132312 ± 23% -57.8% 55881 ± 13% sched_debug.cpu.nr_switches.stddev 0.51 ± 7% +34.8% 0.68 ± 9% perf-stat.i.MPKI 1.04 ± 6% +0.2 1.22 perf-stat.i.branch-miss-rate% 13019839 ± 12% +29.3% 16836662 ± 7% perf-stat.i.cache-misses 3.37 ± 3% +31.3% 4.42 perf-stat.i.cpi 7.645e+10 ± 16% +28.6% 9.829e+10 ± 4% perf-stat.i.cpu-cycles 0.07 ± 6% +0.0 0.09 ± 5% perf-stat.i.dTLB-load-miss-rate% 0.01 ± 27% -0.0 0.01 ± 16% perf-stat.i.dTLB-store-miss-rate% 0.33 ± 3% -20.9% 0.26 perf-stat.i.ipc 1.19 ± 16% +28.6% 1.54 ± 4% perf-stat.i.metric.GHz 5953184 ± 17% +35.2% 8050438 ± 7% perf-stat.i.node-load-misses 21.85 ± 5% +2.9 24.75 ± 3% perf-stat.i.node-store-miss-rate% 907395 ± 12% +61.3% 1463258 ± 8% perf-stat.i.node-store-misses 5141037 ± 13% +18.9% 6111881 ± 5% perf-stat.i.node-stores 0.59 ± 8% +26.7% 0.74 ± 9% perf-stat.overall.MPKI 1.01 +0.2 1.16 perf-stat.overall.branch-miss-rate% 3.40 ± 2% +26.9% 4.32 perf-stat.overall.cpi 0.08 ± 2% +0.0 0.09 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.29 ± 2% -21.2% 0.23 perf-stat.overall.ipc 15.04 ± 2% +4.2 19.29 ± 3% perf-stat.overall.node-store-miss-rate% 12852451 ± 12% +28.9% 16572483 ± 7% perf-stat.ps.cache-misses 1.89e+08 ± 17% +22.8% 2.321e+08 ± 4% perf-stat.ps.cache-references 7.545e+10 ± 16% +28.3% 9.682e+10 ± 4% perf-stat.ps.cpu-cycles 5883748 ± 16% +34.8% 7932536 ± 7% perf-stat.ps.node-load-misses 895770 ± 12% +60.7% 1439876 ± 8% perf-stat.ps.node-store-misses 5068727 ± 13% +18.6% 6012349 ± 5% perf-stat.ps.node-stores 1.596e+12 ± 2% -9.5% 1.444e+12 perf-stat.total.instructions 35.92 -11.4 24.50 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 35.43 -11.3 24.14 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 35.41 -11.3 24.13 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 35.37 -11.3 24.10 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 21.72 -7.4 14.32 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 20.29 -6.9 13.35 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary 20.14 -6.9 13.25 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry 18.70 -6.5 12.23 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle 7.88 -2.1 5.76 ± 2% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 13.72 ± 2% -2.1 11.62 perf-profile.calltrace.cycles-pp.write 13.37 ± 2% -2.1 11.30 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 13.32 ± 2% -2.1 11.27 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 3.53 ± 18% -1.9 1.58 ± 34% perf-profile.calltrace.cycles-pp.record__finish_output.__cmd_record 3.54 ± 18% -1.9 1.59 ± 34% perf-profile.calltrace.cycles-pp.__cmd_record 3.53 ± 18% -1.9 1.58 ± 34% perf-profile.calltrace.cycles-pp.perf_session__process_events.record__finish_output.__cmd_record 3.51 ± 18% -1.9 1.58 ± 34% perf-profile.calltrace.cycles-pp.reader__read_event.perf_session__process_events.record__finish_output.__cmd_record 13.02 ± 3% -1.9 11.10 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 12.88 ± 3% -1.9 10.98 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 7.15 ± 2% -1.9 5.28 ± 2% perf-profile.calltrace.cycles-pp.__flush_smp_call_function_queue.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary 12.57 ± 3% -1.8 10.76 perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 12.39 ± 5% -1.7 10.68 ± 4% perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64 12.07 ± 5% -1.6 10.43 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read 6.18 ± 2% -1.5 4.64 ± 3% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.__flush_smp_call_function_queue.flush_smp_call_function_queue.do_idle.cpu_startup_entry 4.84 ± 2% -1.5 3.38 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 4.76 ± 2% -1.4 3.34 perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary 5.88 -1.4 4.46 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 6.00 -1.4 4.62 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write 6.21 -1.4 4.86 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write 4.83 -1.2 3.64 ± 3% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read 4.96 ± 2% -1.1 3.82 ± 3% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.__flush_smp_call_function_queue.flush_smp_call_function_queue.do_idle 4.87 ± 3% -1.1 3.74 ± 3% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.sched_ttwu_pending.__flush_smp_call_function_queue.flush_smp_call_function_queue 4.48 -1.1 3.37 ± 3% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read 4.18 ± 3% -1.0 3.13 ± 3% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.sched_ttwu_pending.__flush_smp_call_function_queue 1.61 ± 20% -1.0 0.63 ± 57% perf-profile.calltrace.cycles-pp.perf_session__process_user_event.reader__read_event.perf_session__process_events.record__finish_output.__cmd_record 1.61 ± 21% -1.0 0.63 ± 58% perf-profile.calltrace.cycles-pp.__ordered_events__flush.perf_session__process_user_event.reader__read_event.perf_session__process_events.record__finish_output 1.57 ± 21% -1.0 0.62 ± 58% perf-profile.calltrace.cycles-pp.perf_session__deliver_event.__ordered_events__flush.perf_session__process_user_event.reader__read_event.perf_session__process_events 3.68 ± 2% -0.9 2.80 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.sched_ttwu_pending 2.02 -0.6 1.37 perf-profile.calltrace.cycles-pp.ttwu_queue_wakelist.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.41 ± 8% -0.5 0.86 ± 8% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.66 -0.5 1.14 perf-profile.calltrace.cycles-pp.__smp_call_single_queue.ttwu_queue_wakelist.try_to_wake_up.autoremove_wake_function.__wake_up_common 1.55 ± 4% -0.5 1.05 ± 7% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_entity.dequeue_task_fair.__schedule.schedule 1.10 ± 3% -0.4 0.66 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule_idle.do_idle.cpu_startup_entry 0.77 -0.3 0.43 ± 44% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.08 -0.3 0.76 perf-profile.calltrace.cycles-pp.llist_add_batch.__smp_call_single_queue.ttwu_queue_wakelist.try_to_wake_up.autoremove_wake_function 1.20 ± 4% -0.3 0.88 ± 4% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule_idle.do_idle.cpu_startup_entry 0.84 -0.3 0.54 ± 2% perf-profile.calltrace.cycles-pp.llist_reverse_order.__flush_smp_call_function_queue.flush_smp_call_function_queue.do_idle.cpu_startup_entry 1.01 ± 11% -0.3 0.72 ± 3% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule 1.04 ± 16% -0.3 0.76 ± 3% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry 0.88 -0.3 0.62 ± 2% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary 1.59 ± 5% -0.2 1.42 ± 4% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.64 ± 2% -0.1 1.53 perf-profile.calltrace.cycles-pp.select_task_rq.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.32 ± 2% -0.1 1.21 ± 3% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule 0.62 ± 2% -0.1 0.57 ± 3% perf-profile.calltrace.cycles-pp.sched_mm_cid_migrate_to.activate_task.ttwu_do_activate.sched_ttwu_pending.__flush_smp_call_function_queue 0.64 -0.1 0.59 ± 3% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.vfs_write.ksys_write.do_syscall_64 0.43 ± 44% +0.1 0.54 ± 2% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 1.15 ± 11% +0.2 1.36 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.vfs_read.ksys_read.do_syscall_64 1.26 ± 4% +0.3 1.54 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.vfs_read.ksys_read.do_syscall_64 0.00 +0.6 0.59 ± 3% perf-profile.calltrace.cycles-pp.select_idle_core.select_idle_cpu.select_idle_sibling.select_task_rq_fair.select_task_rq 0.00 +0.6 0.62 ± 4% perf-profile.calltrace.cycles-pp.select_idle_cpu.select_idle_sibling.select_task_rq_fair.select_task_rq.try_to_wake_up 0.00 +0.7 0.66 ± 3% perf-profile.calltrace.cycles-pp.__list_del_entry_valid_or_report.finish_wait.pipe_read.vfs_read.ksys_read 0.78 ± 2% +0.9 1.69 perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_write.vfs_write.ksys_write.do_syscall_64 0.86 ± 9% +1.2 2.01 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.vfs_write.ksys_write 3.02 ± 2% +1.2 4.18 perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.pipe_read.vfs_read.ksys_read 0.74 ± 11% +1.2 1.90 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.vfs_write 0.00 +1.2 1.22 perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.pipe_write.vfs_write.ksys_write 7.36 ± 4% +1.7 9.04 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.prepare_to_wait_event.pipe_read.vfs_read.ksys_read 3.10 ± 9% +1.7 4.82 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_read.vfs_read.ksys_read.do_syscall_64 2.58 ± 13% +1.8 4.36 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.pipe_read.vfs_read.ksys_read 11.12 ± 8% +1.9 13.01 perf-profile.calltrace.cycles-pp.prepare_to_wait_event.pipe_read.vfs_read.ksys_read.do_syscall_64 6.10 ± 5% +1.9 8.02 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.prepare_to_wait_event.pipe_read.vfs_read 2.55 ± 4% +5.5 8.08 ± 3% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.pipe_read.vfs_read.ksys_read 4.49 ± 6% +5.7 10.16 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.finish_wait.pipe_read.vfs_read 4.81 ± 3% +5.7 10.56 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.finish_wait.pipe_read.vfs_read.ksys_read 5.02 ± 5% +6.2 11.25 perf-profile.calltrace.cycles-pp.finish_wait.pipe_read.vfs_read.ksys_read.do_syscall_64 6.74 ± 3% +7.6 14.36 ± 2% perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_read.vfs_read.ksys_read.do_syscall_64 45.33 +15.6 60.97 perf-profile.calltrace.cycles-pp.read 44.47 +15.8 60.32 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 44.33 +15.8 60.18 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 43.54 +15.9 59.43 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 43.29 +16.0 59.28 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 42.94 +16.1 59.02 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 35.92 -11.4 24.50 perf-profile.children.cycles-pp.secondary_startup_64_no_verify 35.92 -11.4 24.50 perf-profile.children.cycles-pp.cpu_startup_entry 35.88 -11.4 24.48 perf-profile.children.cycles-pp.do_idle 35.43 -11.3 24.14 perf-profile.children.cycles-pp.start_secondary 22.04 -7.5 14.54 perf-profile.children.cycles-pp.cpuidle_idle_call 20.58 -7.0 13.55 perf-profile.children.cycles-pp.cpuidle_enter 20.47 -7.0 13.48 perf-profile.children.cycles-pp.cpuidle_enter_state 18.95 -6.5 12.41 perf-profile.children.cycles-pp.intel_idle 16.98 ± 3% -3.1 13.86 ± 3% perf-profile.children.cycles-pp.__schedule 8.00 -2.1 5.85 ± 2% perf-profile.children.cycles-pp.flush_smp_call_function_queue 13.81 ± 2% -2.1 11.68 perf-profile.children.cycles-pp.write 3.53 ± 18% -1.9 1.58 ± 34% perf-profile.children.cycles-pp.record__finish_output 3.53 ± 18% -1.9 1.58 ± 34% perf-profile.children.cycles-pp.perf_session__process_events 3.56 ± 18% -1.9 1.62 ± 34% perf-profile.children.cycles-pp.__cmd_record 3.52 ± 18% -1.9 1.58 ± 34% perf-profile.children.cycles-pp.reader__read_event 13.06 ± 3% -1.9 11.12 perf-profile.children.cycles-pp.ksys_write 12.91 ± 3% -1.9 11.00 perf-profile.children.cycles-pp.vfs_write 7.34 -1.9 5.43 ± 2% perf-profile.children.cycles-pp.__flush_smp_call_function_queue 12.60 ± 3% -1.8 10.78 perf-profile.children.cycles-pp.pipe_write 12.40 ± 5% -1.7 10.69 ± 4% perf-profile.children.cycles-pp.schedule 6.34 -1.6 4.78 ± 2% perf-profile.children.cycles-pp.sched_ttwu_pending 4.93 ± 2% -1.5 3.44 perf-profile.children.cycles-pp.schedule_idle 5.90 -1.4 4.47 ± 2% perf-profile.children.cycles-pp.try_to_wake_up 6.01 -1.4 4.62 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function 6.22 -1.4 4.87 ± 2% perf-profile.children.cycles-pp.__wake_up_common 5.10 ± 2% -1.2 3.93 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate 4.86 -1.2 3.69 ± 3% perf-profile.children.cycles-pp.dequeue_task_fair 5.02 ± 2% -1.1 3.88 ± 3% perf-profile.children.cycles-pp.activate_task 4.53 -1.1 3.44 ± 3% perf-profile.children.cycles-pp.dequeue_entity 4.34 ± 2% -1.1 3.28 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair 3.04 ± 6% -1.0 2.01 ± 7% perf-profile.children.cycles-pp.update_cfs_group 3.85 ± 2% -0.9 2.93 ± 4% perf-profile.children.cycles-pp.enqueue_entity 1.61 ± 20% -0.9 0.72 ± 35% perf-profile.children.cycles-pp.__ordered_events__flush 1.61 ± 20% -0.9 0.72 ± 35% perf-profile.children.cycles-pp.perf_session__process_user_event 1.58 ± 21% -0.9 0.71 ± 35% perf-profile.children.cycles-pp.perf_session__deliver_event 1.24 ± 14% -0.7 0.53 ± 27% perf-profile.children.cycles-pp.process_simple 1.19 ± 25% -0.7 0.49 ± 43% perf-profile.children.cycles-pp.evlist__parse_sample 2.03 -0.6 1.38 perf-profile.children.cycles-pp.ttwu_queue_wakelist 2.25 ± 4% -0.6 1.63 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 1.02 ± 26% -0.6 0.45 ± 44% perf-profile.children.cycles-pp.evsel__parse_sample 0.90 ± 14% -0.5 0.36 ± 23% perf-profile.children.cycles-pp.ordered_events__queue 0.87 ± 14% -0.5 0.35 ± 24% perf-profile.children.cycles-pp.queue_event 1.67 -0.5 1.14 perf-profile.children.cycles-pp.__smp_call_single_queue 1.17 ± 3% -0.4 0.80 ± 3% perf-profile.children.cycles-pp.update_curr 0.71 ± 2% -0.4 0.34 ± 31% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 3.38 ± 2% -0.4 3.00 ± 3% perf-profile.children.cycles-pp.update_load_avg 1.54 ± 10% -0.3 1.20 ± 2% perf-profile.children.cycles-pp.prepare_task_switch 1.23 ± 4% -0.3 0.91 ± 4% perf-profile.children.cycles-pp.switch_mm_irqs_off 1.08 -0.3 0.76 perf-profile.children.cycles-pp.llist_add_batch 0.86 -0.3 0.56 ± 2% perf-profile.children.cycles-pp.llist_reverse_order 0.91 -0.3 0.64 ± 3% perf-profile.children.cycles-pp.menu_select 0.42 ± 6% -0.2 0.19 ± 8% perf-profile.children.cycles-pp.copy_page_from_iter 0.39 ± 3% -0.2 0.17 ± 30% perf-profile.children.cycles-pp.perf_tp_event 0.38 ± 6% -0.2 0.17 ± 9% perf-profile.children.cycles-pp._copy_from_iter 0.38 ± 4% -0.2 0.18 ± 5% perf-profile.children.cycles-pp.cpuacct_charge 0.54 ± 3% -0.2 0.34 perf-profile.children.cycles-pp.update_rq_clock_task 0.56 -0.2 0.37 perf-profile.children.cycles-pp.call_function_single_prep_ipi 1.06 ± 3% -0.2 0.88 perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.50 ± 4% -0.2 0.34 ± 3% perf-profile.children.cycles-pp.cpus_share_cache 0.78 -0.2 0.62 ± 2% perf-profile.children.cycles-pp.__switch_to 0.74 ± 2% -0.1 0.59 perf-profile.children.cycles-pp.__switch_to_asm 0.41 ± 2% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.native_sched_clock 0.41 ± 4% -0.1 0.28 ± 4% perf-profile.children.cycles-pp.set_next_entity 0.26 ± 25% -0.1 0.13 ± 46% perf-profile.children.cycles-pp.evsel__parse_sample_timestamp 0.37 ± 4% -0.1 0.24 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.33 ± 3% -0.1 0.21 ± 4% perf-profile.children.cycles-pp.ktime_get 1.65 ± 2% -0.1 1.53 perf-profile.children.cycles-pp.select_task_rq 0.80 -0.1 0.69 ± 5% perf-profile.children.cycles-pp.___perf_sw_event 0.32 ± 3% -0.1 0.21 ± 4% perf-profile.children.cycles-pp.sched_clock 1.09 -0.1 0.98 ± 2% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.24 ± 20% -0.1 0.13 ± 23% perf-profile.children.cycles-pp.build_id__mark_dso_hit 0.68 ± 3% -0.1 0.58 ± 2% perf-profile.children.cycles-pp.finish_task_switch 1.00 -0.1 0.90 ± 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.30 ± 3% -0.1 0.21 perf-profile.children.cycles-pp.update_rq_clock 0.21 ± 4% -0.1 0.13 perf-profile.children.cycles-pp.tick_nohz_idle_enter 0.27 ± 3% -0.1 0.20 ± 4% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.10 ± 20% -0.1 0.03 ±100% perf-profile.children.cycles-pp.thread__find_map 0.14 ± 6% -0.1 0.08 ± 26% perf-profile.children.cycles-pp.perf_trace_buf_update 0.20 ± 2% -0.1 0.13 ± 5% perf-profile.children.cycles-pp.security_file_permission 0.19 ± 3% -0.1 0.12 ± 8% perf-profile.children.cycles-pp.intel_idle_irq 0.18 ± 3% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.__wrgsbase_inactive 0.18 ± 4% -0.1 0.11 ± 5% perf-profile.children.cycles-pp.read_tsc 0.32 ± 17% -0.1 0.26 ± 7% perf-profile.children.cycles-pp.update_blocked_averages 0.19 ± 3% -0.1 0.13 ± 5% perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.19 ± 3% -0.1 0.14 ± 3% perf-profile.children.cycles-pp.reweight_entity 0.64 ± 2% -0.1 0.59 ± 2% perf-profile.children.cycles-pp.sched_mm_cid_migrate_to 0.15 ± 4% -0.1 0.09 ± 7% perf-profile.children.cycles-pp.local_clock_noinstr 0.19 ± 19% -0.1 0.13 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.16 ± 2% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack 0.10 ± 18% -0.1 0.05 ± 50% perf-profile.children.cycles-pp.machine__findnew_thread 0.09 ± 5% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime 0.12 ± 19% -0.1 0.07 ± 23% perf-profile.children.cycles-pp.machines__deliver_event 0.12 ± 3% -0.0 0.07 ± 27% perf-profile.children.cycles-pp.tracing_gen_ctx_irq_test 0.18 ± 6% -0.0 0.13 ± 2% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.16 ± 2% -0.0 0.11 ± 7% perf-profile.children.cycles-pp.apparmor_file_permission 0.13 ± 5% -0.0 0.08 ± 4% perf-profile.children.cycles-pp.get_cpu_device 0.24 ± 4% -0.0 0.20 ± 4% perf-profile.children.cycles-pp.perf_adjust_freq_unthr_context 0.72 ± 2% -0.0 0.67 ± 2% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.13 ± 3% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.place_entity 0.25 ± 5% -0.0 0.20 ± 4% perf-profile.children.cycles-pp.perf_event_task_tick 0.07 ± 10% -0.0 0.02 ± 99% perf-profile.children.cycles-pp.inode_needs_update_time 0.71 -0.0 0.67 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt 0.27 ± 5% -0.0 0.23 ± 10% perf-profile.children.cycles-pp.__update_load_avg_se 0.08 ± 13% -0.0 0.04 ± 72% perf-profile.children.cycles-pp.do_fault 0.08 ± 13% -0.0 0.04 ± 72% perf-profile.children.cycles-pp.do_read_fault 0.62 ± 2% -0.0 0.58 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.15 ± 4% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.avg_vruntime 0.10 ± 6% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.cpuidle_get_cpu_driver 0.19 ± 3% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.__fdget_pos 0.11 ± 3% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.ct_idle_exit 0.10 ± 7% -0.0 0.06 ± 19% perf-profile.children.cycles-pp.exc_page_fault 0.10 ± 10% -0.0 0.06 ± 19% perf-profile.children.cycles-pp.do_user_addr_fault 0.10 ± 4% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.__rdgsbase_inactive 0.10 ± 10% -0.0 0.07 ± 16% perf-profile.children.cycles-pp.asm_exc_page_fault 0.15 ± 3% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.tick_nohz_next_event 0.10 ± 9% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.irq_enter_rcu 0.09 ± 11% -0.0 0.06 ± 20% perf-profile.children.cycles-pp.handle_mm_fault 0.17 ± 4% -0.0 0.14 ± 6% perf-profile.children.cycles-pp.__fget_light 0.09 -0.0 0.06 ± 6% perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event 0.15 ± 19% -0.0 0.12 ± 6% perf-profile.children.cycles-pp.__update_blocked_fair 0.11 ± 4% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.touch_atime 0.10 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.tick_irq_enter 0.09 ± 9% -0.0 0.06 ± 20% perf-profile.children.cycles-pp.__handle_mm_fault 0.09 -0.0 0.06 perf-profile.children.cycles-pp.ct_kernel_enter 0.10 ± 5% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.atime_needs_update 0.09 ± 6% -0.0 0.06 perf-profile.children.cycles-pp.ct_kernel_exit_state 0.08 ± 5% -0.0 0.06 ± 9% perf-profile.children.cycles-pp.file_update_time 0.07 ± 5% -0.0 0.04 ± 44% perf-profile.children.cycles-pp.pick_eevdf 0.18 ± 5% -0.0 0.15 ± 7% perf-profile.children.cycles-pp.nohz_run_idle_balance 0.08 ± 11% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.__dequeue_entity 0.10 ± 3% -0.0 0.07 ± 8% perf-profile.children.cycles-pp.call_cpuidle 0.09 ± 4% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.__calc_delta 0.11 ± 6% -0.0 0.09 ± 8% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.18 ± 3% -0.0 0.16 ± 6% perf-profile.children.cycles-pp.__cgroup_account_cputime 0.10 ± 3% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.08 ± 6% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.12 ± 4% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.task_work_run 0.14 ± 5% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.08 ± 5% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.poll_idle 0.13 ± 6% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.exit_to_user_mode_loop 0.10 ± 8% -0.0 0.08 ± 4% perf-profile.children.cycles-pp.wake_affine 0.10 ± 5% -0.0 0.08 ± 4% perf-profile.children.cycles-pp.sysvec_call_function_single 0.09 ± 5% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.09 ± 4% -0.0 0.08 perf-profile.children.cycles-pp.kill_fasync 0.09 ± 5% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.attach_entity_load_avg 0.30 ± 2% +0.0 0.32 ± 2% perf-profile.children.cycles-pp.remove_entity_load_avg 0.25 ± 4% +0.0 0.28 ± 2% perf-profile.children.cycles-pp.restore_fpregs_from_fpstate 0.07 ± 7% +0.0 0.10 ± 8% perf-profile.children.cycles-pp.task_tick_fair 0.41 ± 2% +0.0 0.45 ± 2% perf-profile.children.cycles-pp.switch_fpu_return 0.06 ± 7% +0.0 0.11 ± 5% perf-profile.children.cycles-pp._find_next_bit 0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.detach_tasks 0.42 ± 9% +0.1 0.49 ± 11% perf-profile.children.cycles-pp.idle_cpu 0.49 ± 5% +0.1 0.60 ± 4% perf-profile.children.cycles-pp.available_idle_cpu 0.34 ± 4% +0.3 0.60 ± 3% perf-profile.children.cycles-pp.select_idle_core 0.26 ± 5% +0.3 0.53 ± 2% perf-profile.children.cycles-pp.osq_unlock 0.36 ± 5% +0.3 0.63 ± 3% perf-profile.children.cycles-pp.select_idle_cpu 2.07 ± 3% +0.3 2.34 perf-profile.children.cycles-pp.mutex_unlock 0.27 ±100% +0.6 0.83 ± 3% perf-profile.children.cycles-pp.__list_del_entry_valid_or_report 3.08 ± 2% +1.2 4.23 perf-profile.children.cycles-pp.mutex_spin_on_owner 11.14 ± 8% +1.9 13.03 perf-profile.children.cycles-pp.prepare_to_wait_event 5.02 ± 5% +6.2 11.26 perf-profile.children.cycles-pp.finish_wait 2.99 ± 3% +6.3 9.31 ± 2% perf-profile.children.cycles-pp.osq_lock 13.95 ± 3% +8.3 22.25 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 7.53 ± 3% +8.5 16.06 ± 2% perf-profile.children.cycles-pp.__mutex_lock 15.76 ± 7% +8.7 24.51 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 57.97 +13.8 71.77 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 57.80 +13.8 71.60 perf-profile.children.cycles-pp.do_syscall_64 45.42 +15.6 61.05 perf-profile.children.cycles-pp.read 43.54 +15.9 59.44 perf-profile.children.cycles-pp.ksys_read 43.31 +16.0 59.30 perf-profile.children.cycles-pp.vfs_read 42.99 +16.1 59.07 perf-profile.children.cycles-pp.pipe_read 18.95 -6.5 12.41 perf-profile.self.cycles-pp.intel_idle 3.00 ± 5% -1.0 2.00 ± 7% perf-profile.self.cycles-pp.update_cfs_group 2.20 ± 5% -0.6 1.56 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 1.01 ± 26% -0.6 0.44 ± 44% perf-profile.self.cycles-pp.evsel__parse_sample 0.86 ± 14% -0.5 0.34 ± 24% perf-profile.self.cycles-pp.queue_event 2.60 ± 6% -0.4 2.18 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.85 ± 7% -0.4 0.48 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irq 1.08 -0.3 0.76 perf-profile.self.cycles-pp.llist_add_batch 1.22 ± 4% -0.3 0.90 ± 4% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.59 ± 28% -0.3 0.28 ± 49% perf-profile.self.cycles-pp.reader__read_event 0.85 -0.3 0.56 ± 2% perf-profile.self.cycles-pp.llist_reverse_order 3.11 -0.3 2.84 perf-profile.self.cycles-pp.prepare_to_wait_event 0.71 ± 3% -0.3 0.46 perf-profile.self.cycles-pp.flush_smp_call_function_queue 1.50 -0.2 1.26 ± 2% perf-profile.self.cycles-pp.__schedule 0.82 ± 19% -0.2 0.58 ± 10% perf-profile.self.cycles-pp.prepare_task_switch 0.52 ± 3% -0.2 0.33 ± 2% perf-profile.self.cycles-pp.update_rq_clock_task 0.37 ± 4% -0.2 0.18 ± 6% perf-profile.self.cycles-pp.cpuacct_charge 0.56 -0.2 0.37 ± 2% perf-profile.self.cycles-pp.call_function_single_prep_ipi 1.05 ± 3% -0.2 0.86 perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 1.87 ± 3% -0.2 1.69 ± 5% perf-profile.self.cycles-pp.update_load_avg 0.21 ± 5% -0.2 0.03 ± 70% perf-profile.self.cycles-pp._copy_from_iter 0.49 ± 3% -0.2 0.34 ± 2% perf-profile.self.cycles-pp.cpus_share_cache 0.24 ± 4% -0.2 0.09 ± 36% perf-profile.self.cycles-pp.perf_tp_event 0.76 -0.2 0.61 ± 2% perf-profile.self.cycles-pp.__switch_to 0.50 ± 7% -0.2 0.35 ± 3% perf-profile.self.cycles-pp.enqueue_task_fair 0.74 ± 2% -0.1 0.59 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 0.74 -0.1 0.60 ± 2% perf-profile.self.cycles-pp.___perf_sw_event 1.43 ± 3% -0.1 1.29 perf-profile.self.cycles-pp.pipe_read 0.39 ± 3% -0.1 0.25 ± 5% perf-profile.self.cycles-pp.update_curr 0.39 ± 3% -0.1 0.26 ± 2% perf-profile.self.cycles-pp.native_sched_clock 0.26 ± 24% -0.1 0.12 ± 47% perf-profile.self.cycles-pp.evsel__parse_sample_timestamp 0.33 ± 26% -0.1 0.20 ± 2% perf-profile.self.cycles-pp.enqueue_entity 0.24 ± 5% -0.1 0.12 ± 29% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template 0.42 ± 2% -0.1 0.30 ± 2% perf-profile.self.cycles-pp.menu_select 0.41 ± 15% -0.1 0.29 ± 7% perf-profile.self.cycles-pp.newidle_balance 0.32 ± 3% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.ttwu_queue_wakelist 0.57 ± 5% -0.1 0.47 ± 5% perf-profile.self.cycles-pp.pipe_write 0.35 ± 8% -0.1 0.24 ± 2% perf-profile.self.cycles-pp.sched_ttwu_pending 0.25 ± 3% -0.1 0.16 ± 6% perf-profile.self.cycles-pp.do_idle 0.31 ± 2% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.dequeue_task_fair 0.19 ± 3% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.reweight_entity 0.17 ± 4% -0.1 0.11 ± 4% perf-profile.self.cycles-pp.__wrgsbase_inactive 0.15 ± 3% -0.1 0.09 ± 6% perf-profile.self.cycles-pp.select_task_rq_fair 0.29 ± 4% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.schedule 0.64 ± 2% -0.1 0.59 ± 2% perf-profile.self.cycles-pp.sched_mm_cid_migrate_to 0.17 ± 4% -0.1 0.11 ± 6% perf-profile.self.cycles-pp.cpuidle_enter_state 0.17 ± 4% -0.1 0.11 ± 5% perf-profile.self.cycles-pp.read_tsc 0.47 ± 4% -0.1 0.41 ± 2% perf-profile.self.cycles-pp.finish_task_switch 0.16 ± 6% -0.1 0.10 ± 10% perf-profile.self.cycles-pp.ktime_get 0.27 ± 4% -0.1 0.22 ± 2% perf-profile.self.cycles-pp.dequeue_entity 0.18 ± 18% -0.1 0.13 ± 3% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.17 ± 5% -0.1 0.12 ± 3% perf-profile.self.cycles-pp.cpuidle_idle_call 0.09 -0.1 0.04 ± 75% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime 0.12 ± 4% -0.0 0.07 ± 29% perf-profile.self.cycles-pp.tracing_gen_ctx_irq_test 0.15 ± 5% -0.0 0.10 ± 3% perf-profile.self.cycles-pp.__flush_smp_call_function_queue 0.15 -0.0 0.10 ± 6% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack 0.13 ± 2% -0.0 0.08 ± 10% perf-profile.self.cycles-pp.intel_idle_irq 0.13 ± 3% -0.0 0.08 perf-profile.self.cycles-pp.get_cpu_device 0.16 ± 3% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.try_to_wake_up 0.14 ± 3% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.vfs_write 0.19 ± 4% -0.0 0.15 ± 4% perf-profile.self.cycles-pp.vfs_read 0.24 ± 4% -0.0 0.19 ± 3% perf-profile.self.cycles-pp.select_idle_sibling 0.10 ± 4% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.cpuidle_enter 0.10 ± 6% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.cpuidle_get_cpu_driver 0.09 ± 5% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.__rdgsbase_inactive 0.11 ± 3% -0.0 0.08 ± 14% perf-profile.self.cycles-pp.write 0.12 ± 3% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.apparmor_file_permission 0.14 ± 6% -0.0 0.11 ± 6% perf-profile.self.cycles-pp.avg_vruntime 0.17 ± 4% -0.0 0.14 ± 5% perf-profile.self.cycles-pp.__fget_light 0.10 ± 5% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.pick_next_task_fair 0.10 ± 5% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.call_cpuidle 0.08 ± 5% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.sched_clock_idle_wakeup_event 0.08 ± 8% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.ct_kernel_exit_state 0.17 ± 6% -0.0 0.14 ± 6% perf-profile.self.cycles-pp.nohz_run_idle_balance 0.11 ± 8% -0.0 0.08 ± 4% perf-profile.self.cycles-pp.perf_adjust_freq_unthr_context 0.08 ± 8% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.__calc_delta 0.14 ± 5% -0.0 0.12 ± 9% perf-profile.self.cycles-pp.__intel_pmu_enable_all 0.07 ± 18% -0.0 0.05 ± 7% perf-profile.self.cycles-pp.__update_blocked_fair 0.09 ± 5% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.update_rq_clock 0.06 ± 7% -0.0 0.05 ± 7% perf-profile.self.cycles-pp.ttwu_do_activate 0.09 +0.0 0.10 ± 4% perf-profile.self.cycles-pp.attach_entity_load_avg 0.25 ± 3% +0.0 0.28 ± 2% perf-profile.self.cycles-pp.restore_fpregs_from_fpstate 0.21 ± 3% +0.0 0.24 perf-profile.self.cycles-pp.__wake_up_common 0.05 ± 7% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.select_idle_core 0.06 ± 11% +0.0 0.11 ± 4% perf-profile.self.cycles-pp._find_next_bit 0.42 ± 9% +0.1 0.48 ± 11% perf-profile.self.cycles-pp.idle_cpu 0.48 ± 5% +0.1 0.60 ± 4% perf-profile.self.cycles-pp.available_idle_cpu 0.26 ± 5% +0.3 0.53 ± 2% perf-profile.self.cycles-pp.osq_unlock 2.04 ± 3% +0.3 2.31 perf-profile.self.cycles-pp.mutex_unlock 0.26 ±100% +0.6 0.82 ± 3% perf-profile.self.cycles-pp.__list_del_entry_valid_or_report 1.20 ± 2% +0.8 1.96 perf-profile.self.cycles-pp.__mutex_lock 3.05 ± 2% +1.2 4.21 perf-profile.self.cycles-pp.mutex_spin_on_owner 2.97 ± 3% +6.3 9.25 ± 2% perf-profile.self.cycles-pp.osq_lock 15.74 ± 7% +8.8 24.49 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath *************************************************************************************************** lkp-icl-2sp9: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase: gcc-12/performance/x86_64-rhel-8.3/100%/debian-11.1-x86_64-20220510.cgz/300s/lkp-icl-2sp9/pipe/unixbench commit: b4bd6b4bac ("fs/pipe: move check to pipe_has_watch_queue()") dfaabf916b ("fs/pipe: remove unnecessary spinlock from pipe_write()") b4bd6b4bac8edd61 dfaabf916b1ca83cfac856745db ---------------- --------------------------- %stddev %change %stddev \ | \ 11392341 ± 4% -15.3% 9645397 ± 5% meminfo.DirectMap2M 42770 ± 24% -57.9% 17999 ±109% numa-numastat.node0.other_node 24.86 ± 3% -1.2 23.67 turbostat.C1% 62.00 ± 2% +7.0% 66.33 turbostat.PkgTmp 8329 ± 5% +9.4% 9113 ± 4% numa-meminfo.node0.KernelStack 1857 ± 4% +16.7% 2167 ± 7% numa-meminfo.node0.Mapped 7896 ± 5% -9.9% 7112 ± 5% numa-meminfo.node1.KernelStack 8329 ± 5% +9.4% 9115 ± 4% numa-vmstat.node0.nr_kernel_stack 42770 ± 24% -57.9% 17999 ±109% numa-vmstat.node0.numa_other 7899 ± 5% -9.9% 7115 ± 5% numa-vmstat.node1.nr_kernel_stack 14.67 ± 8% -29.5% 10.33 ± 12% perf-c2c.DRAM.local 73.33 ± 18% +21.4% 89.00 ± 6% perf-c2c.DRAM.remote 96.00 ± 6% +14.2% 109.67 ± 3% perf-c2c.HITM.local 52.67 ± 18% +23.4% 65.00 ± 2% perf-c2c.HITM.remote 9019 +1.0% 9112 proc-vmstat.nr_mapped 1997539 -1.2% 1974166 proc-vmstat.numa_hit 1931156 -1.2% 1907811 proc-vmstat.numa_local 2118929 -1.1% 2094589 proc-vmstat.pgalloc_normal 2026002 -1.3% 2000324 proc-vmstat.pgfree 15820 ± 9% +20.1% 19006 ± 9% sched_debug.cfs_rq:/.load.avg 23187 ± 40% +60.4% 37191 ± 11% sched_debug.cfs_rq:/.load.stddev 0.09 ± 5% +10.0% 0.10 ± 8% sched_debug.cfs_rq:/.nr_running.stddev 6.45 ± 18% +32.1% 8.53 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.avg 0.00 ± 17% -42.4% 0.00 ± 19% sched_debug.cpu.next_balance.stddev 50406 -16.5% 42082 ± 13% sched_debug.cpu.nr_switches.max 88653 +1.1% 89668 unixbench.score 73491273 ± 70% +51.8% 1.115e+08 unixbench.throughput 1453075 -1.1% 1436609 unixbench.time.minor_page_faults 4802 +1.2% 4862 unixbench.time.percent_of_cpu_this_job_got 2578 +2.5% 2643 unixbench.time.user_time 4.312e+10 +1.1% 4.362e+10 unixbench.workload 0.08 ± 7% -6.6% 0.08 perf-stat.i.MPKI 0.20 ± 2% -0.0 0.18 ± 2% perf-stat.i.branch-miss-rate% 6918885 ± 8% -32.8% 4651719 ± 5% perf-stat.i.branch-misses 580993 -2.1% 568663 perf-stat.i.cache-misses 1.31 ± 2% -3.9% 1.25 perf-stat.i.cpi 514928 +5.8% 545029 perf-stat.i.cycles-between-cache-misses 26064 ± 2% -6.0% 24489 ± 2% perf-stat.i.node-loads 22.62 ± 2% +11.2 33.77 perf-stat.i.node-store-miss-rate% 32126 ± 4% +24.8% 40087 ± 2% perf-stat.i.node-store-misses 128256 ± 2% -11.0% 114161 ± 3% perf-stat.i.node-stores 25.57 -1.2 24.34 perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 38.33 -0.9 37.39 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 35.13 -0.9 34.21 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 41.26 -0.8 40.42 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 42.88 -0.8 42.04 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 53.55 -0.7 52.86 perf-profile.calltrace.cycles-pp.write 2.99 -0.1 2.92 perf-profile.calltrace.cycles-pp.atime_needs_update.touch_atime.pipe_read.vfs_read.ksys_read 1.53 -0.0 1.49 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.86 -0.0 1.83 perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.57 +0.0 0.59 perf-profile.calltrace.cycles-pp.__cond_resched.mutex_lock.pipe_read.vfs_read.ksys_read 0.69 +0.0 0.71 perf-profile.calltrace.cycles-pp.__get_task_ioprio.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.81 +0.0 0.83 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write 0.63 +0.0 0.66 perf-profile.calltrace.cycles-pp.aa_file_perm.apparmor_file_permission.security_file_permission.vfs_read.ksys_read 1.94 +0.0 1.97 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.vfs_read.ksys_read.do_syscall_64 1.23 +0.0 1.27 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.vfs_read.ksys_read.do_syscall_64 0.57 ± 2% +0.0 0.60 perf-profile.calltrace.cycles-pp.__cond_resched.mutex_lock.pipe_write.vfs_write.ksys_write 0.72 +0.0 0.76 perf-profile.calltrace.cycles-pp.timestamp_truncate.inode_needs_update_time.file_update_time.pipe_write.vfs_write 1.95 +0.0 2.00 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.vfs_write.ksys_write.do_syscall_64 1.17 +0.0 1.22 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 3.74 +0.1 3.80 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64 1.25 +0.1 1.31 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 0.92 ± 2% +0.1 1.01 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.read 6.45 +0.1 6.54 perf-profile.calltrace.cycles-pp.__entry_text_start.read 6.17 +0.1 6.28 perf-profile.calltrace.cycles-pp.__entry_text_start.write 7.20 +0.1 7.31 perf-profile.calltrace.cycles-pp._copy_from_iter.copy_page_from_iter.pipe_write.vfs_write.ksys_write 1.05 +0.1 1.16 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.write 2.20 +0.1 2.32 perf-profile.calltrace.cycles-pp.apparmor_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 6.97 +0.1 7.09 perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.pipe_read.vfs_read.ksys_read 2.22 ± 3% +0.1 2.35 perf-profile.calltrace.cycles-pp.inode_needs_update_time.file_update_time.pipe_write.vfs_write.ksys_write 2.72 ± 2% +0.1 2.86 perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.vfs_write.ksys_write.do_syscall_64 8.13 +0.1 8.27 perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.vfs_read.ksys_read.do_syscall_64 2.85 +0.1 3.00 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.14 ± 2% +0.1 2.28 perf-profile.calltrace.cycles-pp.apparmor_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 8.77 +0.2 8.93 perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.vfs_write.ksys_write.do_syscall_64 2.79 +0.2 2.96 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 20.55 +0.2 20.75 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.17 ±141% +0.4 0.52 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 34.24 +0.5 34.71 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 31.03 +0.5 31.50 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 37.08 +0.5 37.61 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 38.65 +0.6 39.21 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 49.34 +0.7 50.07 perf-profile.calltrace.cycles-pp.read 2.86 -1.4 1.44 perf-profile.children.cycles-pp._raw_spin_lock_irq 26.42 -1.3 25.10 perf-profile.children.cycles-pp.pipe_write 35.58 -0.9 34.65 perf-profile.children.cycles-pp.vfs_write 38.61 -0.9 37.68 perf-profile.children.cycles-pp.ksys_write 53.84 -0.7 53.16 perf-profile.children.cycles-pp.write 79.41 -0.3 79.11 perf-profile.children.cycles-pp.do_syscall_64 81.80 -0.3 81.54 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 3.54 -0.1 3.48 perf-profile.children.cycles-pp.atime_needs_update 3.88 -0.0 3.83 perf-profile.children.cycles-pp.touch_atime 0.25 -0.0 0.24 perf-profile.children.cycles-pp.secondary_startup_64_no_verify 0.25 -0.0 0.24 perf-profile.children.cycles-pp.cpu_startup_entry 0.25 -0.0 0.24 perf-profile.children.cycles-pp.do_idle 0.25 -0.0 0.23 ± 2% perf-profile.children.cycles-pp.start_secondary 0.89 +0.0 0.91 perf-profile.children.cycles-pp.__wake_up_common 0.56 +0.0 0.58 perf-profile.children.cycles-pp.rcu_all_qs 0.26 ± 6% +0.0 0.29 perf-profile.children.cycles-pp.tick_sched_timer 0.84 +0.0 0.87 perf-profile.children.cycles-pp.rw_verify_area 0.64 +0.0 0.67 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 2.66 +0.0 2.70 perf-profile.children.cycles-pp.mutex_unlock 5.71 +0.0 5.75 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack 1.64 +0.0 1.68 perf-profile.children.cycles-pp.__get_task_ioprio 1.43 +0.0 1.48 perf-profile.children.cycles-pp.aa_file_perm 1.12 +0.0 1.17 perf-profile.children.cycles-pp.exit_to_user_mode_prepare 1.50 +0.1 1.56 perf-profile.children.cycles-pp.__cond_resched 1.66 +0.1 1.72 perf-profile.children.cycles-pp.timestamp_truncate 4.22 +0.1 4.29 perf-profile.children.cycles-pp.__wake_up_common_lock 7.55 +0.1 7.64 perf-profile.children.cycles-pp._copy_from_iter 4.22 +0.1 4.32 perf-profile.children.cycles-pp.mutex_lock 7.26 +0.1 7.36 perf-profile.children.cycles-pp.__entry_text_start 3.09 +0.1 3.20 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 7.15 +0.1 7.27 perf-profile.children.cycles-pp._copy_to_iter 2.54 ± 3% +0.1 2.68 perf-profile.children.cycles-pp.inode_needs_update_time 8.30 +0.1 8.45 perf-profile.children.cycles-pp.copy_page_to_iter 2.96 ± 2% +0.1 3.11 perf-profile.children.cycles-pp.file_update_time 9.11 +0.2 9.27 perf-profile.children.cycles-pp.copy_page_from_iter 2.16 +0.2 2.36 perf-profile.children.cycles-pp.syscall_return_via_sysret 21.38 +0.2 21.61 perf-profile.children.cycles-pp.pipe_read 4.77 +0.3 5.05 perf-profile.children.cycles-pp.apparmor_file_permission 6.15 +0.3 6.45 perf-profile.children.cycles-pp.security_file_permission 34.50 +0.5 34.96 perf-profile.children.cycles-pp.ksys_read 31.53 +0.5 32.01 perf-profile.children.cycles-pp.vfs_read 49.55 +0.7 50.29 perf-profile.children.cycles-pp.read 2.69 -1.3 1.36 perf-profile.self.cycles-pp._raw_spin_lock_irq 5.28 -0.4 4.90 perf-profile.self.cycles-pp.pipe_write 1.88 -0.1 1.79 perf-profile.self.cycles-pp.atime_needs_update 0.16 +0.0 0.17 perf-profile.self.cycles-pp.__wake_up_sync_key 1.02 +0.0 1.04 perf-profile.self.cycles-pp.ksys_read 0.49 +0.0 0.51 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.72 +0.0 0.75 perf-profile.self.cycles-pp.__wake_up_common 1.16 +0.0 1.19 perf-profile.self.cycles-pp.syscall_exit_to_user_mode 1.39 +0.0 1.42 perf-profile.self.cycles-pp.__get_task_ioprio 1.27 +0.0 1.30 perf-profile.self.cycles-pp._copy_to_iter 2.70 +0.0 2.73 perf-profile.self.cycles-pp.mutex_lock 5.52 +0.0 5.56 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack 1.32 +0.0 1.35 perf-profile.self.cycles-pp.__wake_up_common_lock 0.94 +0.0 0.98 perf-profile.self.cycles-pp.__cond_resched 2.54 +0.0 2.58 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.08 +0.0 1.12 perf-profile.self.cycles-pp.ksys_write 0.95 +0.1 1.00 perf-profile.self.cycles-pp.exit_to_user_mode_prepare 1.66 +0.1 1.71 perf-profile.self.cycles-pp.copy_page_from_iter 1.36 +0.1 1.42 perf-profile.self.cycles-pp.timestamp_truncate 2.09 +0.1 2.16 perf-profile.self.cycles-pp._copy_from_iter 5.46 +0.1 5.56 perf-profile.self.cycles-pp.vfs_read 1.37 ± 4% +0.1 1.49 perf-profile.self.cycles-pp.inode_needs_update_time 2.15 +0.2 2.33 perf-profile.self.cycles-pp.syscall_return_via_sysret 5.21 +0.2 5.40 perf-profile.self.cycles-pp.vfs_write 3.25 ± 2% +0.3 3.51 perf-profile.self.cycles-pp.apparmor_file_permission *************************************************************************************************** lkp-csl-d02: 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 128G memory ========================================================================================= class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime: memory/gcc-12/performance/x86_64-rhel-8.3/1/debian-11.1-x86_64-20220510.cgz/lkp-csl-d02/pipeherd/stress-ng/60s commit: b4bd6b4bac ("fs/pipe: move check to pipe_has_watch_queue()") dfaabf916b ("fs/pipe: remove unnecessary spinlock from pipe_write()") b4bd6b4bac8edd61 dfaabf916b1ca83cfac856745db ---------------- --------------------------- %stddev %change %stddev \ | \ 41401292 -32.3% 28028489 ± 3% cpuidle..usage 5.49 ± 16% +4.9 10.35 ± 15% mpstat.cpu.all.sys% 7049 ± 8% +22.0% 8603 ± 12% perf-c2c.HITM.local 7049 ± 8% +22.0% 8603 ± 12% perf-c2c.HITM.total 2.67 ± 15% +58.2% 4.22 ± 16% vmstat.procs.r 973076 ± 15% -24.2% 737453 ± 16% vmstat.system.cs 1155970 ± 7% -24.4% 874362 ± 7% meminfo.Active 1155822 ± 7% -24.4% 874215 ± 7% meminfo.Active(anon) 1276090 ± 5% -21.0% 1007947 ± 5% meminfo.Shmem 288959 ± 7% -24.4% 218555 ± 7% proc-vmstat.nr_active_anon 1006691 -6.7% 939642 proc-vmstat.nr_file_pages 319042 ± 5% -21.0% 251993 ± 5% proc-vmstat.nr_shmem 288959 ± 7% -24.4% 218555 ± 7% proc-vmstat.nr_zone_active_anon 672348 -11.1% 598000 ± 3% proc-vmstat.numa_hit 672345 -11.7% 593890 ± 2% proc-vmstat.numa_local 711347 ± 2% -11.6% 629037 ± 2% proc-vmstat.pgalloc_normal 632945 -34.2% 416502 ± 4% stress-ng.pipeherd.context_switches_per_sec 37968269 -34.2% 24976673 ± 4% stress-ng.pipeherd.ops 632676 -34.2% 416194 ± 4% stress-ng.pipeherd.ops_per_sec 8449 ± 5% +49.8% 12656 ± 8% stress-ng.time.involuntary_context_switches 261.33 +60.2% 418.67 ± 9% stress-ng.time.percent_of_cpu_this_job_got 158.27 +62.1% 256.62 ± 9% stress-ng.time.system_time 37968591 -34.2% 24977642 ± 4% stress-ng.time.voluntary_context_switches 392.00 ± 12% +39.3% 546.00 ± 12% turbostat.Avg_MHz 11.14 ± 6% +3.6 14.73 ± 8% turbostat.Busy% 13053767 -66.8% 4330713 ± 10% turbostat.C1 17.57 ± 15% -8.9 8.68 ± 19% turbostat.C1% 27519662 -15.6% 23233631 ± 2% turbostat.C1E 0.16 ± 2% -31.6% 0.11 ± 8% turbostat.IPC 86032 -27.2% 62641 ± 3% turbostat.POLL 78281965 ± 8% +33.4% 1.044e+08 ± 14% perf-stat.i.cache-references 1010492 ± 15% -23.7% 770793 ± 16% perf-stat.i.context-switches 1.458e+10 ± 12% +37.3% 2.002e+10 ± 12% perf-stat.i.cpu-cycles 11457 ± 15% +44.2% 16519 ± 15% perf-stat.i.cpu-migrations 1162226 ± 3% -12.0% 1022908 ± 6% perf-stat.i.dTLB-load-misses 0.45 ± 12% -21.0% 0.35 ± 14% perf-stat.i.ipc 0.04 ± 77% +138.8% 0.10 ± 46% perf-stat.i.major-faults 0.40 ± 12% +37.3% 0.56 ± 12% perf-stat.i.metric.GHz 2.47 ± 7% -0.7 1.76 ± 15% perf-stat.overall.cache-miss-rate% 1.98 +45.1% 2.88 ± 8% perf-stat.overall.cpi 7598 ± 12% +47.2% 11182 ± 15% perf-stat.overall.cycles-between-cache-misses 34.28 +2.6 36.83 perf-stat.overall.iTLB-load-miss-rate% 0.50 -30.6% 0.35 ± 8% perf-stat.overall.ipc 77236506 ± 8% +33.2% 1.028e+08 ± 14% perf-stat.ps.cache-references 996570 ± 15% -23.8% 759052 ± 16% perf-stat.ps.context-switches 1.438e+10 ± 12% +37.1% 1.972e+10 ± 12% perf-stat.ps.cpu-cycles 11299 ± 15% +44.0% 16269 ± 15% perf-stat.ps.cpu-migrations 1147284 ± 3% -12.1% 1008110 ± 6% perf-stat.ps.dTLB-load-misses 0.04 ± 77% +138.3% 0.10 ± 46% perf-stat.ps.major-faults 5.686e+11 ± 2% -16.8% 4.73e+11 ± 3% perf-stat.total.instructions 11798 +79.7% 21202 ± 12% sched_debug.cfs_rq:/.avg_vruntime.avg 35545 ± 4% +24.8% 44364 ± 10% sched_debug.cfs_rq:/.avg_vruntime.max 7825 ± 2% +115.8% 16887 ± 15% sched_debug.cfs_rq:/.avg_vruntime.min 0.24 ± 11% +26.9% 0.31 ± 10% sched_debug.cfs_rq:/.h_nr_running.avg 0.39 ± 3% +20.1% 0.46 ± 7% sched_debug.cfs_rq:/.h_nr_running.stddev 19291 ± 11% +117.4% 41944 ± 37% sched_debug.cfs_rq:/.load.avg 11798 +79.7% 21202 ± 12% sched_debug.cfs_rq:/.min_vruntime.avg 35545 ± 4% +24.8% 44364 ± 10% sched_debug.cfs_rq:/.min_vruntime.max 7825 ± 2% +115.8% 16886 ± 15% sched_debug.cfs_rq:/.min_vruntime.min 0.24 ± 11% +26.9% 0.31 ± 10% sched_debug.cfs_rq:/.nr_running.avg 0.39 ± 3% +20.1% 0.46 ± 7% sched_debug.cfs_rq:/.nr_running.stddev 4.00 ± 36% +206.2% 12.25 ± 52% sched_debug.cfs_rq:/.runnable_avg.min 193.19 ± 10% +28.5% 248.32 ± 10% sched_debug.cfs_rq:/.runnable_avg.stddev 3.83 ± 32% +217.4% 12.17 ± 53% sched_debug.cfs_rq:/.util_avg.min 192.68 ± 10% +28.2% 247.04 ± 9% sched_debug.cfs_rq:/.util_avg.stddev 227345 +12.7% 256128 ± 5% sched_debug.cpu.avg_idle.avg 12384 ± 17% +47.8% 18301 ± 24% sched_debug.cpu.avg_idle.min 746.38 ± 12% +30.0% 970.32 ± 6% sched_debug.cpu.curr->pid.avg 3592 ± 4% +6.8% 3838 sched_debug.cpu.curr->pid.max 1237 ± 11% +21.2% 1500 ± 2% sched_debug.cpu.curr->pid.stddev 0.24 ± 11% +36.5% 0.33 ± 6% sched_debug.cpu.nr_running.avg 0.37 ± 7% +36.5% 0.50 ± 9% sched_debug.cpu.nr_running.stddev 1030752 -33.1% 689795 ± 4% sched_debug.cpu.nr_switches.avg 1160111 -34.4% 761385 ± 5% sched_debug.cpu.nr_switches.max 919187 -31.5% 629289 ± 4% sched_debug.cpu.nr_switches.min 60982 ± 8% -51.5% 29571 ± 12% sched_debug.cpu.nr_switches.stddev 0.00 ± 35% +200.0% 0.00 ± 20% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi 0.01 ± 23% +6255.9% 0.72 ± 64% perf-sched.sch_delay.max.ms.__cond_resched.mutex_lock.pipe_read.vfs_read.ksys_read 0.01 ± 5% +2144.0% 0.19 ±212% perf-sched.sch_delay.max.ms.__cond_resched.pipe_read.vfs_read.ksys_read.do_syscall_64 0.02 ± 7% +212.3% 0.08 ± 77% perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep 0.01 ± 35% +362.5% 0.02 ± 51% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi 0.49 +21.1% 0.59 ± 4% perf-sched.total_wait_and_delay.average.ms 1872181 -18.2% 1531048 ± 4% perf-sched.total_wait_and_delay.count.ms 0.49 +21.1% 0.59 ± 4% perf-sched.total_wait_time.average.ms 5.79 ±104% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt 0.30 +21.9% 0.37 ± 4% perf-sched.wait_and_delay.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64 149.15 ± 2% -22.0% 116.38 ± 11% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm 8.33 ± 24% -76.0% 2.00 ± 95% perf-sched.wait_and_delay.count.__cond_resched.shmem_get_folio_gfp.shmem_write_begin.generic_perform_write.shmem_file_write_iter 1.33 ± 35% -100.0% 0.00 perf-sched.wait_and_delay.count.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt 1793631 -19.1% 1451839 ± 4% perf-sched.wait_and_delay.count.pipe_read.vfs_read.ksys_read.do_syscall_64 1264 +27.5% 1611 ± 10% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm 6.40 ± 89% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt 0.28 +12.8% 0.32 ± 5% perf-sched.wait_time.avg.ms.__cond_resched.__mutex_lock.constprop.0.pipe_read 0.09 ±141% +236.4% 0.31 ± 20% perf-sched.wait_time.avg.ms.__cond_resched.__mutex_lock.constprop.0.pipe_write 5.39 ± 72% -93.5% 0.35 ±223% perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock.__fdget_pos.ksys_write.do_syscall_64 0.28 +16.4% 0.32 ± 3% perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock.pipe_read.vfs_read.ksys_read 0.26 ± 4% +19.6% 0.31 ± 4% perf-sched.wait_time.avg.ms.__cond_resched.pipe_read.vfs_read.ksys_read.do_syscall_64 0.27 +129.6% 0.62 ±105% perf-sched.wait_time.avg.ms.__cond_resched.task_work_run.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode 5.79 ±104% -99.9% 0.01 ±166% perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt 0.30 +21.9% 0.36 ± 4% perf-sched.wait_time.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64 149.11 ± 2% -22.0% 116.35 ± 11% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm 0.31 ± 5% +19.4% 0.36 ± 4% perf-sched.wait_time.max.ms.__cond_resched.__mutex_lock.constprop.0.pipe_read 0.09 ±141% +278.6% 0.34 ± 7% perf-sched.wait_time.max.ms.__cond_resched.__mutex_lock.constprop.0.pipe_write 10.55 ± 65% -96.7% 0.35 ±223% perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.__fdget_pos.ksys_write.do_syscall_64 0.32 ± 2% +90.7% 0.61 ± 64% perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.pipe_read.vfs_read.ksys_read 0.31 +21.1% 0.37 ± 6% perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.pipe_write.vfs_write.ksys_write 0.30 +27.2% 0.39 ± 6% perf-sched.wait_time.max.ms.__cond_resched.pipe_read.vfs_read.ksys_read.do_syscall_64 0.29 ± 2% +1656.1% 5.10 ±207% perf-sched.wait_time.max.ms.__cond_resched.task_work_run.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode 6.40 ± 89% -99.8% 0.01 ±188% perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt 0.32 +23.0% 0.39 ± 4% perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.constprop.0.pipe_read 0.28 +25.0% 0.36 ± 5% perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.constprop.0.pipe_write 0.01 ± 20% -33.3% 0.01 ± 19% perf-sched.wait_time.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open 68.52 -10.6 57.90 ± 2% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 66.83 -10.5 56.34 ± 2% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 66.82 -10.5 56.33 ± 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 66.75 -10.4 56.30 ± 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 15.28 -9.6 5.69 ± 11% perf-profile.calltrace.cycles-pp.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle 59.97 -7.4 52.61 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 57.88 -6.6 51.32 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary 59.25 -6.6 52.69 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry 3.53 -1.6 1.88 ± 8% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 3.19 -1.5 1.69 ± 8% perf-profile.calltrace.cycles-pp.__flush_smp_call_function_queue.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary 3.22 -1.3 1.96 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64 3.11 -1.2 1.91 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read 2.43 -1.1 1.30 ± 8% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.__flush_smp_call_function_queue.flush_smp_call_function_queue.do_idle.cpu_startup_entry 2.44 ± 2% -1.1 1.34 ± 8% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write 7.77 -1.1 6.68 ± 3% perf-profile.calltrace.cycles-pp.write 2.46 ± 4% -1.1 1.37 ± 9% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 2.29 -1.0 1.25 ± 8% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 2.33 ± 4% -1.0 1.30 ± 9% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary 2.63 ± 2% -1.0 1.64 ± 8% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write 7.42 -0.9 6.47 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 7.36 -0.9 6.44 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 7.26 -0.9 6.38 ± 3% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 7.13 -0.8 6.30 ± 3% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 2.81 -0.8 2.03 ± 4% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call 6.82 -0.7 6.13 ± 3% perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.52 -0.7 0.85 ± 8% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.__flush_smp_call_function_queue.flush_smp_call_function_queue.do_idle 1.52 -0.6 0.87 ± 6% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read 1.42 ± 2% -0.6 0.79 ± 8% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.sched_ttwu_pending.__flush_smp_call_function_queue.flush_smp_call_function_queue 1.34 ± 2% -0.6 0.73 ± 7% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.sched_ttwu_pending.__flush_smp_call_function_queue 1.38 -0.6 0.80 ± 6% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read 1.00 ± 2% -0.5 0.47 ± 45% perf-profile.calltrace.cycles-pp.ttwu_queue_wakelist.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.16 ± 2% -0.5 0.64 ± 9% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.sched_ttwu_pending 1.52 ± 3% -0.5 1.00 ± 7% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary 0.63 ± 6% -0.1 0.54 ± 2% perf-profile.calltrace.cycles-pp.__list_add_valid_or_report.prepare_to_wait_event.pipe_read.vfs_read.ksys_read 0.63 +0.3 0.90 ± 4% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.vfs_read.ksys_read.do_syscall_64 0.83 +0.4 1.18 ± 3% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.vfs_read.ksys_read.do_syscall_64 1.80 ± 4% +0.9 2.71 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_read.vfs_read.ksys_read.do_syscall_64 0.00 +0.9 0.95 ± 2% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.pipe_write.vfs_write.ksys_write 1.47 ± 4% +1.0 2.45 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.pipe_read.vfs_read.ksys_read 0.71 ± 3% +1.0 1.73 ± 8% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.vfs_write 1.38 +1.1 2.44 ± 2% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.pipe_read.vfs_read.ksys_read 0.86 +1.1 1.94 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.vfs_write.ksys_write 0.34 ± 70% +1.2 1.50 ± 4% perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_write.vfs_write.ksys_write.do_syscall_64 6.45 ± 2% +2.0 8.49 ± 8% perf-profile.calltrace.cycles-pp.prepare_to_wait_event.pipe_read.vfs_read.ksys_read.do_syscall_64 3.03 ± 5% +2.1 5.10 ± 16% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.prepare_to_wait_event.pipe_read.vfs_read 4.01 ± 2% +2.1 6.11 ± 13% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.prepare_to_wait_event.pipe_read.vfs_read.ksys_read 41.50 +3.6 45.08 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle 2.42 +3.9 6.30 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.finish_wait.pipe_read.vfs_read 2.79 +4.0 6.74 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.finish_wait.pipe_read.vfs_read.ksys_read 3.02 +4.2 7.23 ± 10% perf-profile.calltrace.cycles-pp.finish_wait.pipe_read.vfs_read.ksys_read.do_syscall_64 1.12 +4.4 5.49 ± 15% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.pipe_read.vfs_read.ksys_read 3.54 +6.3 9.82 ± 8% perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_read.vfs_read.ksys_read.do_syscall_64 22.75 +12.1 34.82 ± 5% perf-profile.calltrace.cycles-pp.read 21.81 +12.5 34.31 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 21.73 +12.5 34.25 ± 5% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 21.43 +12.6 33.99 ± 5% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 21.31 +12.6 33.92 ± 5% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 21.01 +12.7 33.75 ± 5% perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 68.52 -10.6 57.90 ± 2% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 68.52 -10.6 57.90 ± 2% perf-profile.children.cycles-pp.cpu_startup_entry 68.46 -10.6 57.88 ± 2% perf-profile.children.cycles-pp.do_idle 66.83 -10.5 56.34 ± 2% perf-profile.children.cycles-pp.start_secondary 16.63 -9.8 6.79 ± 10% perf-profile.children.cycles-pp.intel_idle_irq 61.48 -7.4 54.08 ± 2% perf-profile.children.cycles-pp.cpuidle_idle_call 59.33 -6.6 52.75 perf-profile.children.cycles-pp.cpuidle_enter 59.31 -6.6 52.74 perf-profile.children.cycles-pp.cpuidle_enter_state 5.65 -2.3 3.35 ± 7% perf-profile.children.cycles-pp.__schedule 3.63 ± 2% -1.7 1.94 ± 8% perf-profile.children.cycles-pp.flush_smp_call_function_queue 3.29 ± 2% -1.5 1.74 ± 8% perf-profile.children.cycles-pp.__flush_smp_call_function_queue 3.32 -1.3 2.04 ± 5% perf-profile.children.cycles-pp.schedule 2.51 -1.2 1.34 ± 8% perf-profile.children.cycles-pp.sched_ttwu_pending 7.85 -1.1 6.72 ± 4% perf-profile.children.cycles-pp.write 2.53 ± 2% -1.1 1.41 ± 9% perf-profile.children.cycles-pp.schedule_idle 2.45 ± 2% -1.1 1.34 ± 8% perf-profile.children.cycles-pp.autoremove_wake_function 2.35 -1.1 1.30 ± 7% perf-profile.children.cycles-pp.try_to_wake_up 2.64 ± 2% -1.0 1.64 ± 8% perf-profile.children.cycles-pp.__wake_up_common 7.26 -0.9 6.38 ± 3% perf-profile.children.cycles-pp.ksys_write 7.14 -0.8 6.30 ± 3% perf-profile.children.cycles-pp.vfs_write 1.62 -0.7 0.91 ± 7% perf-profile.children.cycles-pp.ttwu_do_activate 6.83 -0.7 6.13 ± 3% perf-profile.children.cycles-pp.pipe_write 1.57 -0.7 0.91 ± 6% perf-profile.children.cycles-pp.dequeue_task_fair 1.50 -0.7 0.84 ± 7% perf-profile.children.cycles-pp.activate_task 1.41 ± 2% -0.6 0.78 ± 7% perf-profile.children.cycles-pp.enqueue_task_fair 1.44 -0.6 0.84 ± 7% perf-profile.children.cycles-pp.dequeue_entity 1.25 ± 2% -0.5 0.70 ± 8% perf-profile.children.cycles-pp.enqueue_entity 1.57 -0.5 1.04 ± 6% perf-profile.children.cycles-pp.menu_select 3.20 ± 4% -0.5 2.73 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irq 1.01 ± 2% -0.5 0.55 ± 9% perf-profile.children.cycles-pp.ttwu_queue_wakelist 2.20 ± 3% -0.4 1.75 ± 3% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 1.08 ± 8% -0.4 0.65 ± 3% perf-profile.children.cycles-pp._raw_spin_lock 0.79 ± 3% -0.3 0.44 ± 9% perf-profile.children.cycles-pp.__smp_call_single_queue 0.88 -0.3 0.53 ± 6% perf-profile.children.cycles-pp.update_load_avg 0.76 ± 4% -0.3 0.41 ± 8% perf-profile.children.cycles-pp.select_task_rq 0.72 ± 5% -0.3 0.39 ± 9% perf-profile.children.cycles-pp.select_task_rq_fair 0.77 ± 5% -0.3 0.46 ± 9% perf-profile.children.cycles-pp.pick_next_task_fair 0.71 ± 2% -0.3 0.41 ± 7% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.61 -0.3 0.34 ± 10% perf-profile.children.cycles-pp.__switch_to_asm 0.60 ± 4% -0.3 0.34 ± 10% perf-profile.children.cycles-pp.llist_add_batch 0.56 ± 3% -0.3 0.30 ± 10% perf-profile.children.cycles-pp.select_idle_sibling 0.61 ± 4% -0.3 0.35 ± 13% perf-profile.children.cycles-pp.prepare_task_switch 0.50 ± 2% -0.2 0.26 ± 5% perf-profile.children.cycles-pp.__switch_to 0.46 ± 2% -0.2 0.26 ± 6% perf-profile.children.cycles-pp.update_curr 0.37 -0.2 0.19 ± 6% perf-profile.children.cycles-pp.native_sched_clock 0.48 ± 5% -0.2 0.30 ± 8% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.36 ± 3% -0.2 0.18 ± 8% perf-profile.children.cycles-pp.cpus_share_cache 0.37 ± 3% -0.2 0.20 ± 10% perf-profile.children.cycles-pp.set_next_entity 0.36 ± 4% -0.2 0.20 ± 13% perf-profile.children.cycles-pp.llist_reverse_order 0.95 ± 7% -0.2 0.80 ± 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.31 ± 4% -0.1 0.17 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu 0.32 -0.1 0.19 ± 11% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.27 -0.1 0.14 ± 9% perf-profile.children.cycles-pp.reweight_entity 0.28 ± 5% -0.1 0.16 ± 6% perf-profile.children.cycles-pp.sched_clock 0.29 ± 3% -0.1 0.17 ± 11% perf-profile.children.cycles-pp.___perf_sw_event 0.27 ± 10% -0.1 0.15 ± 16% perf-profile.children.cycles-pp.__entry_text_start 0.25 ± 3% -0.1 0.14 ± 13% perf-profile.children.cycles-pp.security_file_permission 0.74 ± 6% -0.1 0.63 ± 3% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.73 ± 6% -0.1 0.62 ± 3% perf-profile.children.cycles-pp.hrtimer_interrupt 0.28 ± 5% -0.1 0.17 ± 11% perf-profile.children.cycles-pp.tick_nohz_next_event 0.66 ± 5% -0.1 0.55 ± 2% perf-profile.children.cycles-pp.__list_add_valid_or_report 0.28 -0.1 0.17 ± 9% perf-profile.children.cycles-pp.__update_load_avg_se 0.21 ± 4% -0.1 0.10 ± 11% perf-profile.children.cycles-pp.copy_page_from_iter 0.29 ± 6% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.update_cfs_group 0.23 ± 7% -0.1 0.12 ± 11% perf-profile.children.cycles-pp.avg_vruntime 0.27 ± 4% -0.1 0.16 ± 8% perf-profile.children.cycles-pp.ktime_get 0.23 ± 9% -0.1 0.13 ± 14% perf-profile.children.cycles-pp.poll_idle 0.19 ± 6% -0.1 0.10 ± 7% perf-profile.children.cycles-pp.tick_nohz_idle_enter 0.61 ± 8% -0.1 0.51 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.17 ± 4% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.__wrgsbase_inactive 0.19 ± 8% -0.1 0.10 ± 10% perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.19 ± 9% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.place_entity 0.22 ± 6% -0.1 0.13 ± 7% perf-profile.children.cycles-pp.update_rq_clock 0.20 ± 4% -0.1 0.12 ± 17% perf-profile.children.cycles-pp.apparmor_file_permission 0.15 -0.1 0.07 ± 7% perf-profile.children.cycles-pp.__enqueue_entity 0.17 ± 7% -0.1 0.09 ± 10% perf-profile.children.cycles-pp.call_function_single_prep_ipi 0.16 ± 11% -0.1 0.08 ± 13% perf-profile.children.cycles-pp._copy_from_iter 0.18 ± 5% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack 0.16 ± 7% -0.1 0.09 ± 14% perf-profile.children.cycles-pp.available_idle_cpu 0.17 ± 2% -0.1 0.09 ± 11% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.10 ± 8% -0.1 0.02 ± 99% perf-profile.children.cycles-pp.update_min_vruntime 0.19 ± 8% -0.1 0.12 ± 8% perf-profile.children.cycles-pp.finish_task_switch 0.10 ± 8% -0.1 0.03 ±100% perf-profile.children.cycles-pp.put_prev_task_fair 0.10 ± 8% -0.1 0.03 ±100% perf-profile.children.cycles-pp.copyin 0.12 ± 6% -0.1 0.05 ± 49% perf-profile.children.cycles-pp.local_clock_noinstr 0.18 ± 7% -0.1 0.11 ± 12% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.12 ± 7% -0.1 0.06 ± 8% perf-profile.children.cycles-pp.call_cpuidle 0.15 ± 14% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.__calc_delta 0.10 ± 16% -0.1 0.03 ± 70% perf-profile.children.cycles-pp.cpuacct_charge 0.21 ± 14% -0.1 0.15 ± 12% perf-profile.children.cycles-pp.clock_nanosleep 0.14 ± 9% -0.1 0.08 ± 20% perf-profile.children.cycles-pp.__fget_light 0.15 ± 6% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.__fdget_pos 0.16 ± 10% -0.1 0.10 ± 10% perf-profile.children.cycles-pp.read_tsc 0.09 ± 9% -0.1 0.03 ±101% perf-profile.children.cycles-pp.__rdgsbase_inactive 0.12 ± 6% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.get_cpu_device 0.17 ± 12% -0.1 0.12 ± 16% perf-profile.children.cycles-pp.newidle_balance 0.09 ± 14% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.touch_atime 0.10 ± 9% -0.0 0.05 ± 45% perf-profile.children.cycles-pp.pick_next_task_idle 0.07 ± 12% -0.0 0.02 ± 99% perf-profile.children.cycles-pp.atime_needs_update 0.15 ± 3% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.15 ± 11% -0.0 0.11 ± 11% perf-profile.children.cycles-pp.__x64_sys_clock_nanosleep 0.14 ± 11% -0.0 0.10 ± 12% perf-profile.children.cycles-pp.common_nsleep 0.14 ± 11% -0.0 0.10 ± 12% perf-profile.children.cycles-pp.hrtimer_nanosleep 0.11 ± 11% -0.0 0.07 ± 14% perf-profile.children.cycles-pp.__dequeue_entity 0.09 ± 5% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.13 ± 10% -0.0 0.09 ± 14% perf-profile.children.cycles-pp.do_nanosleep 0.07 ± 7% -0.0 0.03 ±100% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.09 ± 13% -0.0 0.06 ± 9% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.11 ± 11% -0.0 0.07 ± 15% perf-profile.children.cycles-pp.ct_kernel_exit_state 0.23 ± 9% -0.0 0.19 ± 8% perf-profile.children.cycles-pp.copy_page_to_iter 0.09 -0.0 0.06 ± 17% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.19 ± 6% +0.2 0.38 ± 3% perf-profile.children.cycles-pp.osq_unlock 0.35 ± 2% +0.2 0.54 ± 2% perf-profile.children.cycles-pp.__list_del_entry_valid_or_report 0.97 +0.2 1.20 ± 4% perf-profile.children.cycles-pp.mutex_unlock 1.28 ± 2% +0.2 1.53 ± 2% perf-profile.children.cycles-pp.mutex_lock 1.41 +1.1 2.50 ± 2% perf-profile.children.cycles-pp.mutex_spin_on_owner 6.46 ± 2% +2.0 8.50 ± 8% perf-profile.children.cycles-pp.prepare_to_wait_event 41.51 +3.6 45.08 perf-profile.children.cycles-pp.intel_idle 3.03 +4.2 7.24 ± 10% perf-profile.children.cycles-pp.finish_wait 1.33 ± 2% +5.1 6.45 ± 13% perf-profile.children.cycles-pp.osq_lock 8.80 ± 4% +6.8 15.58 ± 12% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 8.21 +6.9 15.12 ± 11% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 4.06 +7.3 11.33 ± 7% perf-profile.children.cycles-pp.__mutex_lock 29.47 +11.5 40.95 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 29.32 +11.5 40.86 ± 3% perf-profile.children.cycles-pp.do_syscall_64 22.84 +12.0 34.88 ± 5% perf-profile.children.cycles-pp.read 21.43 +12.6 33.99 ± 5% perf-profile.children.cycles-pp.ksys_read 21.31 +12.6 33.93 ± 5% perf-profile.children.cycles-pp.vfs_read 21.03 +12.7 33.78 ± 5% perf-profile.children.cycles-pp.pipe_read 16.11 -9.6 6.48 ± 10% perf-profile.self.cycles-pp.intel_idle_irq 1.07 ± 9% -0.4 0.63 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 0.59 ± 8% -0.3 0.28 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.70 ± 2% -0.3 0.40 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.61 -0.3 0.34 ± 10% perf-profile.self.cycles-pp.__switch_to_asm 0.60 ± 4% -0.3 0.34 ± 10% perf-profile.self.cycles-pp.llist_add_batch 0.87 ± 2% -0.3 0.61 ± 6% perf-profile.self.cycles-pp.menu_select 0.48 ± 2% -0.2 0.25 ± 5% perf-profile.self.cycles-pp.__switch_to 0.52 -0.2 0.30 ± 11% perf-profile.self.cycles-pp.__schedule 0.43 ± 6% -0.2 0.22 ± 7% perf-profile.self.cycles-pp.__flush_smp_call_function_queue 0.38 ± 3% -0.2 0.17 ± 10% perf-profile.self.cycles-pp.sched_ttwu_pending 0.36 -0.2 0.18 ± 7% perf-profile.self.cycles-pp.native_sched_clock 0.35 ± 2% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.cpus_share_cache 0.36 ± 4% -0.2 0.19 ± 12% perf-profile.self.cycles-pp.llist_reverse_order 0.34 ± 4% -0.1 0.20 ± 9% perf-profile.self.cycles-pp.flush_smp_call_function_queue 0.33 ± 6% -0.1 0.19 ± 15% perf-profile.self.cycles-pp.prepare_task_switch 0.32 -0.1 0.18 ± 11% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.27 -0.1 0.14 ± 9% perf-profile.self.cycles-pp.reweight_entity 0.26 ± 4% -0.1 0.13 ± 9% perf-profile.self.cycles-pp.enqueue_entity 0.26 ± 3% -0.1 0.14 ± 10% perf-profile.self.cycles-pp.do_idle 0.28 ± 2% -0.1 0.17 ± 8% perf-profile.self.cycles-pp.update_load_avg 0.66 ± 5% -0.1 0.55 ± 2% perf-profile.self.cycles-pp.__list_add_valid_or_report 0.22 ± 5% -0.1 0.12 ± 9% perf-profile.self.cycles-pp.avg_vruntime 0.25 ± 3% -0.1 0.15 ± 12% perf-profile.self.cycles-pp.___perf_sw_event 0.26 -0.1 0.17 ± 8% perf-profile.self.cycles-pp.__update_load_avg_se 0.18 ± 6% -0.1 0.09 ± 15% perf-profile.self.cycles-pp.ttwu_queue_wakelist 0.28 ± 6% -0.1 0.18 ± 8% perf-profile.self.cycles-pp.update_cfs_group 0.17 ± 4% -0.1 0.08 ± 14% perf-profile.self.cycles-pp.__wrgsbase_inactive 0.16 ± 7% -0.1 0.08 ± 16% perf-profile.self.cycles-pp.available_idle_cpu 0.19 ± 8% -0.1 0.11 ± 15% perf-profile.self.cycles-pp.poll_idle 0.17 ± 7% -0.1 0.09 ± 10% perf-profile.self.cycles-pp.call_function_single_prep_ipi 0.18 ± 15% -0.1 0.10 ± 18% perf-profile.self.cycles-pp.cpuidle_idle_call 0.20 ± 4% -0.1 0.12 ± 5% perf-profile.self.cycles-pp.cpuidle_enter_state 0.21 ± 2% -0.1 0.13 ± 12% perf-profile.self.cycles-pp.update_curr 0.14 -0.1 0.06 ± 11% perf-profile.self.cycles-pp.__enqueue_entity 0.18 ± 7% -0.1 0.10 ± 16% perf-profile.self.cycles-pp.apparmor_file_permission 0.17 ± 4% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack 0.15 ± 11% -0.1 0.08 ± 22% perf-profile.self.cycles-pp.dequeue_entity 0.12 ± 7% -0.1 0.06 ± 8% perf-profile.self.cycles-pp.call_cpuidle 0.13 ± 10% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair 0.11 ± 8% -0.1 0.05 ± 46% perf-profile.self.cycles-pp.schedule_idle 0.10 ± 12% -0.1 0.03 ± 70% perf-profile.self.cycles-pp.cpuacct_charge 0.14 ± 11% -0.1 0.08 ± 19% perf-profile.self.cycles-pp.__calc_delta 0.17 ± 7% -0.1 0.10 ± 9% perf-profile.self.cycles-pp.finish_task_switch 0.09 ± 14% -0.1 0.02 ± 99% perf-profile.self.cycles-pp.schedule 0.13 ± 6% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.__fget_light 0.09 ± 9% -0.1 0.03 ±101% perf-profile.self.cycles-pp.__rdgsbase_inactive 0.16 ± 8% -0.1 0.10 ± 10% perf-profile.self.cycles-pp.read_tsc 0.15 ± 8% -0.1 0.09 ± 20% perf-profile.self.cycles-pp.read 0.12 ± 6% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.get_cpu_device 0.13 ± 3% -0.1 0.08 ± 14% perf-profile.self.cycles-pp.vfs_read 0.08 ± 5% -0.1 0.03 ±100% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.13 ± 3% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.vfs_write 0.12 ± 4% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.__entry_text_start 0.13 ± 3% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.dequeue_task_fair 0.13 ± 6% -0.1 0.08 ± 12% perf-profile.self.cycles-pp.write 0.15 ± 9% -0.1 0.10 ± 15% perf-profile.self.cycles-pp.pick_next_task_fair 0.14 ± 3% -0.1 0.09 ± 17% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.12 ± 6% -0.1 0.07 ± 13% perf-profile.self.cycles-pp.select_task_rq_fair 0.11 ± 14% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.try_to_wake_up 0.11 ± 4% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.ktime_get 0.13 ± 18% -0.0 0.09 ± 14% perf-profile.self.cycles-pp.newidle_balance 0.09 ± 15% -0.0 0.05 ± 46% perf-profile.self.cycles-pp.__dequeue_entity 0.11 ± 11% -0.0 0.07 ± 15% perf-profile.self.cycles-pp.ct_kernel_exit_state 0.10 ± 4% -0.0 0.07 ± 15% perf-profile.self.cycles-pp.stress_pipeherd_read_write 0.09 -0.0 0.06 ± 17% perf-profile.self.cycles-pp.__intel_pmu_enable_all 0.19 ± 4% +0.1 0.29 ± 7% perf-profile.self.cycles-pp.__wake_up_common 0.19 ± 6% +0.2 0.37 ± 2% perf-profile.self.cycles-pp.osq_unlock 0.35 ± 2% +0.2 0.54 ± 2% perf-profile.self.cycles-pp.__list_del_entry_valid_or_report 0.96 +0.2 1.19 ± 4% perf-profile.self.cycles-pp.mutex_unlock 1.23 ± 2% +0.3 1.49 ± 2% perf-profile.self.cycles-pp.mutex_lock 1.13 ± 2% +0.9 2.00 ± 3% perf-profile.self.cycles-pp.__mutex_lock 1.40 +1.1 2.48 ± 2% perf-profile.self.cycles-pp.mutex_spin_on_owner 41.51 +3.6 45.08 perf-profile.self.cycles-pp.intel_idle 1.32 ± 2% +5.1 6.43 ± 13% perf-profile.self.cycles-pp.osq_lock 8.77 ± 4% +6.8 15.57 ± 12% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
