hi, Yang Shi,
On Fri, Jan 31, 2025 at 10:38:03AM -0800, Yang Shi wrote:
>
>
>
> On 1/27/25 7:14 PM, kernel test robot wrote:
> > hi, All,
> >
> > we don't have enough knowledge to understand fully the discussion for this
> > patch, we saw "NACK" but there were more discussions later.
> > so below report is just FYI what we observed in our tests. thanks
>
> Thanks for the report. It was nack'ed because of the change to smaps/maps
> files in proc.
>
> >
> > Hello,
> >
> > kernel test robot noticed a 858.5% improvement of vm-scalability.throughput on:
> >
> >
> > commit: 7143ee2391f1ea15e6791e129870473543634de2 ("[PATCH] /dev/zero: make private mapping full anonymous mapping")
> > url: https://github.com/intel-lab-lkp/linux/commits/Yang-Shi/dev-zero-make-private-mapping-full-anonymous-mapping/20250114-063339
> > base: https://git.kernel.org/cgit/linux/kernel/git/gregkh/char-misc.git a68d3cbfade64392507302f3a920113b60dc811f
> > patch link: https://lore.kernel.org/all/20250113223033.4054534-1-yang@xxxxxxxxxxxxxxxxxxxxxx/
> > patch subject: [PATCH] /dev/zero: make private mapping full anonymous mapping
> >
> > testcase: vm-scalability
> > config: x86_64-rhel-9.4
> > compiler: gcc-12
> > test machine: 224 threads 4 sockets Intel(R) Xeon(R) Platinum 8380H CPU @ 2.90GHz (Cooper Lake) with 192G memory
> > parameters:
> >
> > runtime: 300s
> > test: small-allocs
>
> It seems this benchmark allocates huge amount of small areas (each area is
> as big as 40K) by mmap'ing /dev/zero.
>
> This patch makes /dev/zero mapping a full anonymous mapping, so the later
> vma_link_file() is actually skipped, which needs acquire file rmap lock then
> insert the mapping into file rmap tree. The below profiling also showed
> this.
> Quoted here so that we don't have to scroll down:
>
> > 95.60 -95.2 0.42 ±113% perf-profile.children.cycles-pp.__mmap
> > 94.14 -93.6 0.54 ±106% perf-profile.children.cycles-pp.__mmap_new_vma
> > 93.79 -93.6 0.21 ±171% perf-profile.children.cycles-pp.vma_link_file
> > 93.40 -93.4 0.00 perf-profile.children.cycles-pp.rwsem_down_write_slowpath
> > 93.33 -93.3 0.00 perf-profile.children.cycles-pp.rwsem_optimistic_spin
> > 93.44 -93.2 0.24 ±178% perf-profile.children.cycles-pp.down_write
> > 94.55 -93.1 1.40 ± 51% perf-profile.children.cycles-pp.ksys_mmap_pgoff
> > 94.25 -93.0 1.30 ± 59% perf-profile.children.cycles-pp.__mmap_region
> > 92.91 -92.9 0.00 perf-profile.children.cycles-pp.osq_lock
> > 94.45 -92.7 1.72 ± 34% perf-profile.children.cycles-pp.do_mmap
> > 94.46 -92.6 1.83 ± 31% perf-profile.children.cycles-pp.vm_mmap_pgoff
>
> It significantly speed up mmap for this benchmark and the rmap lock
> contention is reduced significantly for both multi-processes and
> multi-threads.
>
> The benchmark itself may exaggerate the improvement, but it may really speed
> up some real life workloads. For example, multiple applications which may
> allocate anonymous mapping by mmap'ing /dev/zero, then they may have
> contention on /dev/zero's rmap lock.
>
> It doesn't make too much sense to link /dev/zero anonymous vmas to the file
> rmap tree. So the below patch should be able to speed up the benchmark too.
sorry for late and thanks a lot for information!
>
> Oliver, can you please give this patch a try?
it seems this is an alternative patch?
since we applied your "/dev/zero: make private mapping full anonymous mapping"
patch upon a68d3cbfad like below:
* 7143ee2391f1e /dev/zero: make private mapping full anonymous mapping
* a68d3cbfade64 memstick: core: fix kernel-doc notation
so I applied below patch also upon a68d3cbfad.
we saw big improvement but not that big.
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase:
gcc-12/performance/x86_64-rhel-9.4/debian-12-x86_64-20240206.cgz/300s/lkp-cpl-4sp2/small-allocs/vm-scalability
commit:
a68d3cbfad ("memstick: core: fix kernel-doc notation")
52ec85cb99 <--- your patch
a68d3cbfade64392 52ec85cb99e9b31dc304eae965a
---------------- ---------------------------
%stddev %change %stddev
\ | \
14364828 ± 4% +410.6% 73349239 ± 3% vm-scalability.throughput
full comparison as below [1] just FYI.
>
>
> diff --git a/mm/vma.c b/mm/vma.c
> index bb2119e5a0d0..1092222c40ae 100644
> --- a/mm/vma.c
> +++ b/mm/vma.c
> @@ -1633,6 +1633,9 @@ static void unlink_file_vma_batch_process(struct
> unlink_vma_file_batch *vb)
> void unlink_file_vma_batch_add(struct unlink_vma_file_batch *vb,
> struct vm_area_struct *vma)
> {
> + if (vma_is_anonymous(vma))
> + return;
> +
> if (vma->vm_file == NULL)
> return;
>
> @@ -1658,6 +1661,9 @@ void unlink_file_vma(struct vm_area_struct *vma)
> {
> struct file *file = vma->vm_file;
>
> + if (vma_is_anonymous(vma))
> + return;
> +
> if (file) {
> struct address_space *mapping = file->f_mapping;
>
> @@ -1672,6 +1678,9 @@ void vma_link_file(struct vm_area_struct *vma)
> struct file *file = vma->vm_file;
> struct address_space *mapping;
>
> + if (vma_is_anonymous(vma))
> + return;
> +
> if (file) {
> mapping = file->f_mapping;
> i_mmap_lock_write(mapping);
>
>
> Because /dev/zero's private mapping is an anonymous mapping with valid
> vm_file, so we need to bail out early if the vma is anonymous even though it
> has vm_file. IMHO, making /dev/zero private mapping a full anonymous mapping
> looks more clean.
>
[1]
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase:
gcc-12/performance/x86_64-rhel-9.4/debian-12-x86_64-20240206.cgz/300s/lkp-cpl-4sp2/small-allocs/vm-scalability
commit:
a68d3cbfad ("memstick: core: fix kernel-doc notation")
52ec85cb99 <--- your patch
a68d3cbfade64392 52ec85cb99e9b31dc304eae965a
---------------- ---------------------------
%stddev %change %stddev
\ | \
5.262e+09 ± 3% -45.0% 2.896e+09 ± 6% cpuidle..time
7924008 ± 3% -79.3% 1643339 ± 11% cpuidle..usage
1871164 ± 4% -22.4% 1452554 ± 12% numa-numastat.node3.local_node
1952164 ± 3% -20.1% 1560294 ± 12% numa-numastat.node3.numa_hit
399.52 -68.2% 126.86 uptime.boot
14507 -15.7% 12232 uptime.idle
6.99 ± 3% +147.9% 17.34 ± 4% vmstat.cpu.id
1.71 +473.6% 9.79 ± 2% vmstat.cpu.us
34204 ± 5% -72.9% 9272 ± 7% vmstat.system.cs
266575 -21.2% 210191 vmstat.system.in
3408 ± 5% -99.8% 8.38 ± 48% perf-c2c.DRAM.local
18076 ± 3% -99.8% 32.25 ± 27% perf-c2c.DRAM.remote
8082 ± 5% -99.8% 15.50 ± 64% perf-c2c.HITM.local
6544 ± 6% -99.8% 13.62 ± 51% perf-c2c.HITM.remote
14627 ± 4% -99.8% 29.12 ± 53% perf-c2c.HITM.total
6.49 ± 3% +8.8 15.24 ± 5% mpstat.cpu.all.idle%
0.63 -0.3 0.32 ± 4% mpstat.cpu.all.irq%
0.03 ± 2% +0.2 0.26 ± 2% mpstat.cpu.all.soft%
91.17 -17.0 74.15 mpstat.cpu.all.sys%
1.68 ± 2% +8.3 10.03 ± 2% mpstat.cpu.all.usr%
337.33 -97.4% 8.88 ± 75% mpstat.max_utilization.seconds
352.76 -77.3% 79.95 ± 2% time.elapsed_time
352.76 -77.3% 79.95 ± 2% time.elapsed_time.max
225965 ± 7% -16.0% 189844 ± 6% time.involuntary_context_switches
9.592e+08 ± 4% +11.9% 1.074e+09 time.minor_page_faults
20852 -8.8% 19012 time.percent_of_cpu_this_job_got
72302 -81.4% 13425 ± 3% time.system_time
1260 ± 3% +41.0% 1777 time.user_time
5393707 ± 5% -98.4% 86880 ± 17% time.voluntary_context_switches
1609925 -50.3% 800493 meminfo.Active
1609925 -50.3% 800493 meminfo.Active(anon)
160837 ± 33% -63.9% 58119 ± 13% meminfo.AnonHugePages
4435665 -18.5% 3614714 meminfo.Cached
1775547 -43.8% 998415 meminfo.Committed_AS
148539 -43.7% 83699 ± 4% meminfo.Mapped
4245538 ± 4% -20.9% 3356561 meminfo.PageTables
14166291 ± 4% -9.6% 12806082 meminfo.SUnreclaim
929777 -88.2% 109274 ± 3% meminfo.Shmem
14315492 ± 4% -9.6% 12947821 meminfo.Slab
64129 ± 4% +418.9% 332751 ± 3% vm-scalability.median
45.40 ± 5% +1961.8 2007 ± 8% vm-scalability.stddev%
14364828 ± 4% +410.6% 73349239 ± 3% vm-scalability.throughput
352.76 -77.3% 79.95 ± 2% vm-scalability.time.elapsed_time
352.76 -77.3% 79.95 ± 2% vm-scalability.time.elapsed_time.max
225965 ± 7% -16.0% 189844 ± 6% vm-scalability.time.involuntary_context_switches
9.592e+08 ± 4% +11.9% 1.074e+09 vm-scalability.time.minor_page_faults
20852 -8.8% 19012 vm-scalability.time.percent_of_cpu_this_job_got
72302 -81.4% 13425 ± 3% vm-scalability.time.system_time
1260 ± 3% +41.0% 1777 vm-scalability.time.user_time
5393707 ± 5% -98.4% 86880 ± 17% vm-scalability.time.voluntary_context_switches
4.316e+09 ± 4% +11.9% 4.832e+09 vm-scalability.workload
265763 ± 4% -20.5% 211398 ± 4% numa-vmstat.node0.nr_page_table_pages
31364 ±106% -85.0% 4690 ±169% numa-vmstat.node0.nr_shmem
12205 ± 67% -74.1% 3161 ±199% numa-vmstat.node1.nr_mapped
265546 ± 4% -21.8% 207742 ± 4% numa-vmstat.node1.nr_page_table_pages
44052 ± 71% -86.0% 6163 ±161% numa-vmstat.node1.nr_shmem
885590 ± 4% -9.9% 797649 ± 4% numa-vmstat.node1.nr_slab_unreclaimable
264589 ± 4% -21.2% 208598 ± 4% numa-vmstat.node2.nr_page_table_pages
881598 ± 4% -10.0% 793829 ± 4% numa-vmstat.node2.nr_slab_unreclaimable
192683 ± 30% -61.0% 75078 ± 70% numa-vmstat.node3.nr_active_anon
286819 ±108% -93.0% 19993 ± 39% numa-vmstat.node3.nr_file_pages
13124 ± 49% -92.3% 1006 ± 57% numa-vmstat.node3.nr_mapped
264499 ± 4% -22.1% 206135 ± 2% numa-vmstat.node3.nr_page_table_pages
139810 ± 14% -90.5% 13229 ± 89% numa-vmstat.node3.nr_shmem
880199 ± 4% -11.8% 776210 ± 5% numa-vmstat.node3.nr_slab_unreclaimable
192683 ± 30% -61.0% 75077 ± 70% numa-vmstat.node3.nr_zone_active_anon
1951359 ± 3% -20.1% 1558936 ± 12% numa-vmstat.node3.numa_hit
1870359 ± 4% -22.4% 1451195 ± 12% numa-vmstat.node3.numa_local
402515 -50.3% 200150 proc-vmstat.nr_active_anon
170568 +1.9% 173746 proc-vmstat.nr_anon_pages
4257257 +0.9% 4296664 proc-vmstat.nr_dirty_background_threshold
8524925 +0.9% 8603835 proc-vmstat.nr_dirty_threshold
1109246 -18.5% 903959 proc-vmstat.nr_file_pages
42815276 +0.9% 43210344 proc-vmstat.nr_free_pages
37525 -43.6% 21164 ± 4% proc-vmstat.nr_mapped
1059932 ± 4% -21.1% 836810 proc-vmstat.nr_page_table_pages
232507 -88.2% 27341 ± 3% proc-vmstat.nr_shmem
37297 -5.0% 35436 proc-vmstat.nr_slab_reclaimable
3537843 ± 4% -9.8% 3192506 proc-vmstat.nr_slab_unreclaimable
402515 -50.3% 200150 proc-vmstat.nr_zone_active_anon
61931 ± 8% -83.8% 10023 ± 45% proc-vmstat.numa_hint_faults
15755 ± 21% -87.1% 2039 ± 97% proc-vmstat.numa_hint_faults_local
6916516 ± 3% -7.1% 6425430 proc-vmstat.numa_hit
6568542 ± 3% -7.5% 6077764 proc-vmstat.numa_local
293942 ± 3% -69.6% 89435 ± 49% proc-vmstat.numa_pte_updates
9.608e+08 ± 4% +11.8% 1.074e+09 proc-vmstat.pgfault
55981 ± 2% -63.1% 20641 ± 2% proc-vmstat.pgreuse
1063552 ± 4% -20.3% 847673 ± 4% numa-meminfo.node0.PageTables
3565610 ± 4% -8.0% 3279375 ± 3% numa-meminfo.node0.SUnreclaim
125455 ±106% -85.2% 18620 ±168% numa-meminfo.node0.Shmem
3592377 ± 4% -7.1% 3336072 ± 4% numa-meminfo.node0.Slab
48482 ± 67% -74.3% 12475 ±199% numa-meminfo.node1.Mapped
1062709 ± 4% -21.7% 831966 ± 4% numa-meminfo.node1.PageTables
3543793 ± 4% -10.0% 3189589 ± 4% numa-meminfo.node1.SUnreclaim
176171 ± 71% -86.0% 24677 ±161% numa-meminfo.node1.Shmem
3593431 ± 4% -10.4% 3220352 ± 4% numa-meminfo.node1.Slab
1058901 ± 4% -21.3% 833124 ± 4% numa-meminfo.node2.PageTables
3527862 ± 4% -10.2% 3168666 ± 5% numa-meminfo.node2.SUnreclaim
3565750 ± 4% -10.3% 3200248 ± 5% numa-meminfo.node2.Slab
770405 ± 30% -61.0% 300435 ± 70% numa-meminfo.node3.Active
770405 ± 30% -61.0% 300435 ± 70% numa-meminfo.node3.Active(anon)
1146977 ±108% -93.0% 80110 ± 40% numa-meminfo.node3.FilePages
52663 ± 47% -91.6% 4397 ± 56% numa-meminfo.node3.Mapped
6368902 ± 20% -21.2% 5021246 ± 2% numa-meminfo.node3.MemUsed
1058539 ± 4% -22.2% 823061 ± 3% numa-meminfo.node3.PageTables
3522496 ± 4% -12.1% 3096728 ± 6% numa-meminfo.node3.SUnreclaim
558943 ± 14% -90.5% 53054 ± 89% numa-meminfo.node3.Shmem
3557392 ± 4% -12.3% 3119454 ± 6% numa-meminfo.node3.Slab
0.82 ± 4% -39.7% 0.50 ± 12% perf-stat.i.MPKI
2.714e+10 ± 2% +185.7% 7.755e+10 ± 6% perf-stat.i.branch-instructions
0.11 ± 3% +0.1 0.20 ± 5% perf-stat.i.branch-miss-rate%
24932893 +156.6% 63980942 ± 5% perf-stat.i.branch-misses
64.93 -10.1 54.87 ± 2% perf-stat.i.cache-miss-rate%
34508 ± 4% -61.4% 13315 ± 10% perf-stat.i.context-switches
7.67 -63.7% 2.79 ± 6% perf-stat.i.cpi
224605 +10.8% 248972 ± 4% perf-stat.i.cpu-clock
696.35 ± 2% -57.4% 296.79 ± 3% perf-stat.i.cpu-migrations
1.102e+11 +128.5% 2.518e+11 ± 6% perf-stat.i.instructions
0.14 +198.2% 0.42 ± 5% perf-stat.i.ipc
24.25 ± 3% +375.8% 115.36 ± 3% perf-stat.i.metric.K/sec
2722043 ± 3% +439.7% 14690226 ± 6% perf-stat.i.minor-faults
2722043 ± 3% +439.7% 14690226 ± 6% perf-stat.i.page-faults
224605 +10.8% 248972 ± 4% perf-stat.i.task-clock
0.81 ± 3% -52.5% 0.39 ± 14% perf-stat.overall.MPKI
0.09 -0.0 0.08 ± 2% perf-stat.overall.branch-miss-rate%
64.81 -6.4 58.40 perf-stat.overall.cache-miss-rate%
7.24 -56.3% 3.17 ± 3% perf-stat.overall.cpi
0.14 +129.0% 0.32 ± 3% perf-stat.overall.ipc
9012 ± 2% -57.5% 3827 perf-stat.overall.path-length
2.701e+10 ± 2% +159.6% 7.012e+10 ± 2% perf-stat.ps.branch-instructions
24708939 +119.2% 54173035 perf-stat.ps.branch-misses
34266 ± 5% -73.9% 8949 ± 7% perf-stat.ps.context-switches
7.941e+11 -9.1% 7.219e+11 perf-stat.ps.cpu-cycles
693.54 ± 2% -68.6% 217.73 ± 5% perf-stat.ps.cpu-migrations
1.097e+11 +108.1% 2.282e+11 ± 2% perf-stat.ps.instructions
2710577 ± 3% +388.7% 13246535 ± 2% perf-stat.ps.minor-faults
2710577 ± 3% +388.7% 13246536 ± 2% perf-stat.ps.page-faults
3.886e+13 ± 2% -52.4% 1.849e+13 perf-stat.total.instructions
64052898 ± 5% -96.2% 2460331 ±166% sched_debug.cfs_rq:/.avg_vruntime.avg
95701822 ± 7% -85.1% 14268127 ±116% sched_debug.cfs_rq:/.avg_vruntime.max
43098762 ± 6% -96.0% 1715136 ±173% sched_debug.cfs_rq:/.avg_vruntime.min
9223270 ± 9% -84.2% 1457904 ±122% sched_debug.cfs_rq:/.avg_vruntime.stddev
0.78 ± 2% -77.0% 0.18 ±130% sched_debug.cfs_rq:/.h_nr_running.avg
43049468 ± 22% -89.3% 4590302 ±180% sched_debug.cfs_rq:/.left_deadline.max
3836405 ± 37% -85.6% 550773 ±176% sched_debug.cfs_rq:/.left_deadline.stddev
43049467 ± 22% -89.3% 4590279 ±180% sched_debug.cfs_rq:/.left_vruntime.max
3836405 ± 37% -85.6% 550772 ±176% sched_debug.cfs_rq:/.left_vruntime.stddev
64052901 ± 5% -96.2% 2460341 ±166% sched_debug.cfs_rq:/.min_vruntime.avg
95701822 ± 7% -85.1% 14268127 ±116% sched_debug.cfs_rq:/.min_vruntime.max
43098762 ± 6% -96.0% 1715136 ±173% sched_debug.cfs_rq:/.min_vruntime.min
9223270 ± 9% -84.2% 1457902 ±122% sched_debug.cfs_rq:/.min_vruntime.stddev
0.77 ± 2% -77.4% 0.17 ±128% sched_debug.cfs_rq:/.nr_running.avg
1.61 ± 24% +396.0% 7.96 ± 62% sched_debug.cfs_rq:/.removed.runnable_avg.avg
86.69 +424.4% 454.62 ± 24% sched_debug.cfs_rq:/.removed.runnable_avg.max
11.14 ± 13% +409.8% 56.79 ± 35% sched_debug.cfs_rq:/.removed.runnable_avg.stddev
1.61 ± 24% +396.0% 7.96 ± 62% sched_debug.cfs_rq:/.removed.util_avg.avg
86.69 +424.4% 454.62 ± 24% sched_debug.cfs_rq:/.removed.util_avg.max
11.14 ± 13% +409.8% 56.79 ± 35% sched_debug.cfs_rq:/.removed.util_avg.stddev
43049467 ± 22% -89.3% 4590282 ±180% sched_debug.cfs_rq:/.right_vruntime.max
3836405 ± 37% -85.6% 550772 ±176% sched_debug.cfs_rq:/.right_vruntime.stddev
286633 ± 43% +262.3% 1038592 ± 36% sched_debug.cfs_rq:/.runnable_avg.avg
34728895 ± 30% +349.2% 1.56e+08 ± 26% sched_debug.cfs_rq:/.runnable_avg.max
2845573 ± 30% +325.9% 12119045 ± 26% sched_debug.cfs_rq:/.runnable_avg.stddev
769.03 -69.9% 231.86 ± 84% sched_debug.cfs_rq:/.util_avg.avg
1621 ± 5% -31.5% 1111 ± 8% sched_debug.cfs_rq:/.util_avg.max
724.17 ± 2% -89.6% 75.66 ±147% sched_debug.cfs_rq:/.util_est.avg
1360 ± 15% -39.2% 826.88 ± 37% sched_debug.cfs_rq:/.util_est.max
766944 ± 3% +18.1% 905901 sched_debug.cpu.avg_idle.avg
321459 ± 2% -35.6% 207172 ± 10% sched_debug.cpu.avg_idle.stddev
195573 -72.7% 53401 ± 24% sched_debug.cpu.clock.avg
195596 -72.7% 53442 ± 24% sched_debug.cpu.clock.max
195548 -72.7% 53352 ± 24% sched_debug.cpu.clock.min
194424 -72.6% 53229 ± 24% sched_debug.cpu.clock_task.avg
194608 -72.6% 53383 ± 24% sched_debug.cpu.clock_task.max
181834 -77.5% 40964 ± 31% sched_debug.cpu.clock_task.min
4241 ± 2% -80.6% 821.65 ±142% sched_debug.cpu.curr->pid.avg
9799 ± 2% -55.4% 4365 ± 17% sched_debug.cpu.curr->pid.max
1365 ± 10% -48.0% 709.44 ± 5% sched_debug.cpu.curr->pid.stddev
537665 ± 4% +31.2% 705318 ± 14% sched_debug.cpu.max_idle_balance_cost.max
3119 ± 56% +579.1% 21184 ± 39% sched_debug.cpu.max_idle_balance_cost.stddev
0.78 ± 2% -76.3% 0.18 ±135% sched_debug.cpu.nr_running.avg
25773 ± 5% -96.1% 1007 ± 41% sched_debug.cpu.nr_switches.avg
48669 ± 10% -76.5% 11448 ± 13% sched_debug.cpu.nr_switches.max
19006 ± 7% -98.6% 258.81 ± 64% sched_debug.cpu.nr_switches.min
4142 ± 8% -66.3% 1396 ± 17% sched_debug.cpu.nr_switches.stddev
0.07 ± 23% -92.9% 0.01 ± 41% sched_debug.cpu.nr_uninterruptible.avg
240.19 ± 16% -82.1% 42.94 ± 41% sched_debug.cpu.nr_uninterruptible.max
-77.92 -88.1% -9.25 sched_debug.cpu.nr_uninterruptible.min
37.87 ± 5% -85.8% 5.36 ± 13% sched_debug.cpu.nr_uninterruptible.stddev
195549 -72.7% 53356 ± 24% sched_debug.cpu_clk
194699 -73.0% 52506 ± 25% sched_debug.ktime
0.00 -100.0% 0.00 sched_debug.rt_rq:.rt_nr_running.avg
0.17 -100.0% 0.00 sched_debug.rt_rq:.rt_nr_running.max
0.01 -100.0% 0.00 sched_debug.rt_rq:.rt_nr_running.stddev
196368 -72.4% 54191 ± 24% sched_debug.sched_clk
0.17 ±142% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__do_fault.do_read_fault.do_pte_missing.__handle_mm_fault
0.19 ± 34% -51.3% 0.09 ± 37% perf-sched.sch_delay.avg.ms.__cond_resched.__kmalloc_cache_noprof.perf_event_mmap_event.perf_event_mmap.__mmap_region
0.14 ± 55% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
0.14 ± 73% -82.5% 0.03 ±168% perf-sched.sch_delay.avg.ms.__cond_resched.change_pud_range.isra.0.change_protection_range
0.11 ± 59% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.vma_link_file.__mmap_new_vma.__mmap_region
0.04 ±132% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write_killable.exec_mmap.begin_new_exec.load_elf_binary
0.02 ± 31% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.dput.__fput.__x64_sys_close.do_syscall_64
0.00 ±223% +51950.0% 0.26 ±212% perf-sched.sch_delay.avg.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
0.25 ± 59% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write
0.12 ±145% -99.1% 0.00 ±141% perf-sched.sch_delay.avg.ms.__cond_resched.mutex_lock.perf_poll.do_poll.constprop
0.25 ± 41% -81.6% 0.05 ± 69% perf-sched.sch_delay.avg.ms.__cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate.isra
0.11 ± 59% -87.1% 0.01 ±198% perf-sched.sch_delay.avg.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.40 ± 50% -97.8% 0.01 ± 30% perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
2.25 ±138% -99.6% 0.01 ± 7% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.32 ±104% -97.3% 0.01 ± 38% perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
0.01 ± 12% -34.9% 0.01 ± 18% perf-sched.sch_delay.avg.ms.irq_thread.kthread.ret_from_fork.ret_from_fork_asm
0.01 ± 20% -100.0% 0.00 perf-sched.sch_delay.avg.ms.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown].[unknown]
0.19 ±185% -95.6% 0.01 ± 44% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.07 ± 20% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.vma_link_file
0.26 ± 17% -98.8% 0.00 ± 10% perf-sched.sch_delay.avg.ms.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
0.03 ± 51% -69.7% 0.01 ± 67% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
0.01 ± 55% +721.9% 0.10 ± 29% perf-sched.sch_delay.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
0.01 ±128% -83.6% 0.00 ± 20% perf-sched.sch_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.vfs_open
0.06 ± 31% +1921.5% 1.23 ±165% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
1.00 ±151% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__do_fault.do_read_fault.do_pte_missing.__handle_mm_fault
25.45 ± 94% -98.6% 0.36 ± 61% perf-sched.sch_delay.max.ms.__cond_resched.__kmalloc_cache_noprof.perf_event_mmap_event.perf_event_mmap.__mmap_region
4.56 ± 67% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
3.55 ± 97% -98.9% 0.04 ±189% perf-sched.sch_delay.max.ms.__cond_resched.change_pud_range.isra.0.change_protection_range
3.16 ± 78% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.vma_link_file.__mmap_new_vma.__mmap_region
0.30 ±159% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write_killable.exec_mmap.begin_new_exec.load_elf_binary
0.03 ± 86% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.dput.__fput.__x64_sys_close.do_syscall_64
0.00 ±223% +3.2e+06% 15.79 ±259% perf-sched.sch_delay.max.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
3.09 ± 45% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write
3.51 ± 21% -86.1% 0.49 ± 72% perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc_noprof.vm_area_alloc.__mmap_new_vma.__mmap_region
3.59 ± 11% -92.0% 0.29 ±165% perf-sched.sch_delay.max.ms.__cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate.isra
1.60 ± 69% -95.7% 0.07 ±243% perf-sched.sch_delay.max.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.81 ± 43% -98.5% 0.01 ± 43% perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
1.02 ± 88% -98.1% 0.02 ± 47% perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
9.68 ± 32% -92.2% 0.76 ± 72% perf-sched.sch_delay.max.ms.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
12.26 ±109% -92.9% 0.87 ±101% perf-sched.sch_delay.max.ms.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
0.03 ±106% -100.0% 0.00 perf-sched.sch_delay.max.ms.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown].[unknown]
37.84 ± 47% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.vma_link_file
4.68 ± 36% -99.8% 0.01 ± 65% perf-sched.sch_delay.max.ms.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
0.36 ±186% -96.3% 0.01 ± 90% perf-sched.sch_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.vfs_open
97903 ± 4% -38.3% 60433 ± 29% perf-sched.total_wait_and_delay.count.ms
3.97 ± 6% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.__kmalloc_cache_noprof.perf_event_mmap_event.perf_event_mmap.__mmap_region
302.41 ± 5% -27.4% 219.54 ± 14% perf-sched.wait_and_delay.avg.ms.irq_thread.kthread.ret_from_fork.ret_from_fork_asm
1.48 ± 6% -90.9% 0.14 ± 79% perf-sched.wait_and_delay.avg.ms.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
327.16 ± 9% -46.6% 174.81 ± 24% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range.do_poll.constprop.0.do_sys_poll
369.37 ± 2% -75.3% 91.05 ± 35% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.96 ± 6% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.vma_link_file
187.66 +120.6% 413.97 ± 14% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
1831 ± 9% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.__kmalloc_cache_noprof.perf_event_mmap_event.perf_event_mmap.__mmap_region
6.17 ± 45% -79.7% 1.25 ±142% perf-sched.wait_and_delay.count.__cond_resched.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
40.50 ± 8% +245.7% 140.00 ± 23% perf-sched.wait_and_delay.count.schedule_hrtimeout_range.do_poll.constprop.0.do_sys_poll
13.17 ± 2% +624.4% 95.38 ± 19% perf-sched.wait_and_delay.count.schedule_hrtimeout_range.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
73021 ± 3% -100.0% 0.00 perf-sched.wait_and_delay.count.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.vma_link_file
11323 ± 3% -75.9% 2725 ± 28% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
1887 ± 45% -96.1% 73.88 ± 78% perf-sched.wait_and_delay.count.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
1238 -34.5% 811.25 ± 13% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
35.19 ± 57% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.__kmalloc_cache_noprof.perf_event_mmap_event.perf_event_mmap.__mmap_region
20.79 ± 19% -95.9% 0.84 ± 93% perf-sched.wait_and_delay.max.ms.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
1240 ± 20% -14.4% 1062 ± 10% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range.do_poll.constprop.0.do_sys_poll
500.34 +31.2% 656.38 ± 39% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
58.83 ± 39% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.vma_link_file
1237 ± 34% +151.7% 3114 ± 25% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
49.27 ±119% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages_noprof.alloc_pages_mpol_noprof.folio_alloc_mpol_noprof.shmem_alloc_folio
58.17 ±187% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__do_fault.do_read_fault.do_pte_missing.__handle_mm_fault
3.78 ± 5% -97.6% 0.09 ± 37% perf-sched.wait_time.avg.ms.__cond_resched.__kmalloc_cache_noprof.perf_event_mmap_event.perf_event_mmap.__mmap_region
2.99 ± 4% +15.4% 3.45 ± 10% perf-sched.wait_time.avg.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
3.92 ± 5% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
4.71 ± 8% -99.5% 0.02 ±170% perf-sched.wait_time.avg.ms.__cond_resched.change_pud_range.isra.0.change_protection_range
1.67 ± 20% -92.7% 0.12 ± 30% perf-sched.wait_time.avg.ms.__cond_resched.down_write.__mmap_new_vma.__mmap_region.do_mmap
2.10 ± 27% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.vma_link_file.__mmap_new_vma.__mmap_region
0.01 ± 44% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write_killable.exec_mmap.begin_new_exec.load_elf_binary
1.67 ± 21% -94.3% 0.10 ± 35% perf-sched.wait_time.avg.ms.__cond_resched.down_write_killable.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
0.04 ±133% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.dput.__fput.__x64_sys_close.do_syscall_64
2.30 ± 14% -95.5% 0.10 ± 42% perf-sched.wait_time.avg.ms.__cond_resched.kmem_cache_alloc_noprof.vm_area_alloc.__mmap_new_vma.__mmap_region
2.00 ± 74% +2917.4% 60.44 ± 33% perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
29.19 ± 5% -38.5% 17.96 ± 28% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.x64_sys_call
0.37 ± 30% +5524.5% 20.95 ± 30% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.do_syscall_64.entry_SYSCALL_64_after_hwframe
302.40 ± 5% -27.4% 219.53 ± 14% perf-sched.wait_time.avg.ms.irq_thread.kthread.ret_from_fork.ret_from_fork_asm
1.40 ± 6% -92.7% 0.10 ± 18% perf-sched.wait_time.avg.ms.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
0.72 ±220% -100.0% 0.00 perf-sched.wait_time.avg.ms.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown].[unknown]
326.84 ± 9% -46.6% 174.54 ± 24% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range.do_poll.constprop.0.do_sys_poll
369.18 ± 2% -75.3% 91.04 ± 35% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.89 ± 6% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.vma_link_file
187.58 +120.6% 413.77 ± 14% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
2.36 ± 29% +1759.6% 43.80 ± 33% perf-sched.wait_time.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
0.01 ±156% -97.9% 0.00 ±264% perf-sched.wait_time.avg.ms.wait_for_partner.fifo_open.do_dentry_open.vfs_open
340.69 ±135% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages_noprof.alloc_pages_mpol_noprof.folio_alloc_mpol_noprof.shmem_alloc_folio
535.09 ±128% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__do_fault.do_read_fault.do_pte_missing.__handle_mm_fault
22.04 ± 32% -98.4% 0.36 ± 61% perf-sched.wait_time.max.ms.__cond_resched.__kmalloc_cache_noprof.perf_event_mmap_event.perf_event_mmap.__mmap_region
13.57 ± 17% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
13.54 ± 10% -99.7% 0.04 ±189% perf-sched.wait_time.max.ms.__cond_resched.change_pud_range.isra.0.change_protection_range
10.17 ± 19% -95.2% 0.49 ± 56% perf-sched.wait_time.max.ms.__cond_resched.down_write.__mmap_new_vma.__mmap_region.do_mmap
11.35 ± 25% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.vma_link_file.__mmap_new_vma.__mmap_region
0.01 ± 32% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write_killable.exec_mmap.begin_new_exec.load_elf_binary
10.62 ± 9% -96.5% 0.38 ± 72% perf-sched.wait_time.max.ms.__cond_resched.down_write_killable.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
0.20 ±199% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.dput.__fput.__x64_sys_close.do_syscall_64
14.42 ± 22% -96.6% 0.49 ± 72% perf-sched.wait_time.max.ms.__cond_resched.kmem_cache_alloc_noprof.vm_area_alloc.__mmap_new_vma.__mmap_region
4.00 ± 74% +19182.5% 772.23 ± 40% perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
10.75 ± 98% +6512.2% 710.88 ± 56% perf-sched.wait_time.max.ms.do_wait.kernel_wait4.do_syscall_64.entry_SYSCALL_64_after_hwframe
15.80 ± 8% -95.2% 0.76 ± 72% perf-sched.wait_time.max.ms.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
11.64 ± 61% -98.9% 0.13 ±132% perf-sched.wait_time.max.ms.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
2.94 ±213% -100.0% 0.00 perf-sched.wait_time.max.ms.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown].[unknown]
1240 ± 20% -14.3% 1062 ± 10% perf-sched.wait_time.max.ms.schedule_hrtimeout_range.do_poll.constprop.0.do_sys_poll
500.11 +31.2% 656.37 ± 39% perf-sched.wait_time.max.ms.schedule_hrtimeout_range.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
32.65 ± 33% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.vma_link_file
1237 ± 34% +151.6% 3113 ± 25% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
95.59 -95.6 0.00 perf-profile.calltrace.cycles-pp.__mmap
95.54 -95.5 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
95.54 -95.5 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__mmap
94.54 -94.5 0.00 perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
94.46 -94.0 0.41 ±138% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
94.14 -93.7 0.40 ±136% perf-profile.calltrace.cycles-pp.__mmap_new_vma.__mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff
93.79 -93.5 0.31 ±134% perf-profile.calltrace.cycles-pp.vma_link_file.__mmap_new_vma.__mmap_region.do_mmap.vm_mmap_pgoff
93.40 -93.4 0.00 perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.vma_link_file.__mmap_new_vma.__mmap_region
93.33 -93.3 0.00 perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.vma_link_file.__mmap_new_vma
93.44 -93.3 0.14 ±264% perf-profile.calltrace.cycles-pp.down_write.vma_link_file.__mmap_new_vma.__mmap_region.do_mmap
94.45 -93.0 1.42 ± 60% perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
94.25 -92.9 1.33 ± 61% perf-profile.calltrace.cycles-pp.__mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
92.89 -92.9 0.00 perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.vma_link_file
0.00 +1.1 1.09 ± 33% perf-profile.calltrace.cycles-pp.dup_mmap.dup_mm.copy_process.kernel_clone.__do_sys_clone
0.00 +1.4 1.37 ± 49% perf-profile.calltrace.cycles-pp.setlocale
0.00 +1.6 1.64 ± 47% perf-profile.calltrace.cycles-pp.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_do_entry
0.00 +1.6 1.64 ± 47% perf-profile.calltrace.cycles-pp.handle_softirqs.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt
0.00 +1.6 1.65 ± 43% perf-profile.calltrace.cycles-pp.do_sys_openat2.__x64_sys_openat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.8 1.76 ± 44% perf-profile.calltrace.cycles-pp.__x64_sys_openat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.9 1.93 ± 26% perf-profile.calltrace.cycles-pp.dup_mm.copy_process.kernel_clone.__do_sys_clone.do_syscall_64
0.00 +2.2 2.16 ± 44% perf-profile.calltrace.cycles-pp.do_pte_missing.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
0.00 +2.2 2.23 ± 33% perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +2.4 2.37 ± 36% perf-profile.calltrace.cycles-pp.zap_present_ptes.zap_pte_range.zap_pmd_range.unmap_page_range.unmap_vmas
0.00 +2.5 2.48 ± 32% perf-profile.calltrace.cycles-pp.get_cpu_sleep_time_us.get_idle_time.uptime_proc_show.seq_read_iter.vfs_read
0.00 +2.5 2.50 ± 45% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.00 +2.5 2.54 ± 47% perf-profile.calltrace.cycles-pp.__mmput.exit_mm.do_exit.do_group_exit.__x64_sys_exit_group
0.00 +2.5 2.54 ± 47% perf-profile.calltrace.cycles-pp.exit_mm.do_exit.do_group_exit.__x64_sys_exit_group.x64_sys_call
0.00 +2.6 2.62 ± 35% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe._Fork
0.00 +2.6 2.62 ± 35% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe._Fork
0.00 +2.6 2.62 ± 35% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe._Fork
0.00 +2.6 2.62 ± 35% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe._Fork
0.00 +2.7 2.68 ± 35% perf-profile.calltrace.cycles-pp.get_idle_time.uptime_proc_show.seq_read_iter.vfs_read.ksys_read
0.00 +2.8 2.77 ± 33% perf-profile.calltrace.cycles-pp.uptime_proc_show.seq_read_iter.vfs_read.ksys_read.do_syscall_64
0.00 +2.8 2.82 ± 32% perf-profile.calltrace.cycles-pp._Fork
0.00 +2.8 2.84 ± 45% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.00 +2.8 2.84 ± 45% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
0.00 +2.9 2.89 ± 39% perf-profile.calltrace.cycles-pp.event_function_call.perf_event_release_kernel.perf_release.__fput.task_work_run
0.00 +2.9 2.89 ± 39% perf-profile.calltrace.cycles-pp.smp_call_function_single.event_function_call.perf_event_release_kernel.perf_release.__fput
0.00 +3.1 3.10 ± 64% perf-profile.calltrace.cycles-pp.proc_reg_read_iter.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.1 3.10 ± 64% perf-profile.calltrace.cycles-pp.seq_read_iter.proc_reg_read_iter.vfs_read.ksys_read.do_syscall_64
0.00 +3.1 3.13 ± 33% perf-profile.calltrace.cycles-pp.asm_exc_page_fault
0.00 +3.2 3.18 ± 37% perf-profile.calltrace.cycles-pp.seq_read_iter.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.2 3.20 ± 28% perf-profile.calltrace.cycles-pp.mutex_unlock.sw_perf_event_destroy._free_event.perf_event_release_kernel.perf_release
0.00 +3.2 3.24 ± 39% perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.2 3.24 ± 36% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.x64_sys_call.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.2 3.24 ± 36% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.x64_sys_call.do_syscall_64
0.00 +3.2 3.24 ± 36% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.x64_sys_call.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.2 3.24 ± 36% perf-profile.calltrace.cycles-pp.x64_sys_call.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.8 3.85 ± 39% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.00 +3.8 3.85 ± 39% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.00 +3.8 3.85 ± 39% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.00 +3.8 3.85 ± 39% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
0.00 +3.8 3.85 ± 39% perf-profile.calltrace.cycles-pp.execve
0.00 +4.0 4.04 ± 43% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.__x64_sys_openat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +4.0 4.04 ± 43% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.__x64_sys_openat.do_syscall_64
0.00 +4.1 4.10 ± 30% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.__mmput.exit_mm
0.00 +4.2 4.18 ± 31% perf-profile.calltrace.cycles-pp.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap.__mmput
0.00 +4.2 4.18 ± 31% perf-profile.calltrace.cycles-pp.zap_pte_range.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap
0.00 +4.2 4.20 ± 28% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.__mmput.exit_mm.do_exit
0.00 +4.2 4.25 ± 65% perf-profile.calltrace.cycles-pp.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write.do_syscall_64
0.00 +4.3 4.27 ± 26% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.00 +4.3 4.30 ± 22% perf-profile.calltrace.cycles-pp.__mmput.exit_mm.do_exit.do_group_exit.get_signal
0.00 +4.3 4.30 ± 22% perf-profile.calltrace.cycles-pp.exit_mm.do_exit.do_group_exit.get_signal.arch_do_signal_or_restart
0.00 +4.5 4.46 ± 59% perf-profile.calltrace.cycles-pp.shmem_file_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +4.6 4.57 ± 58% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write.writen
0.00 +4.7 4.68 ± 55% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write.writen.record__pushfn
0.00 +4.7 4.68 ± 55% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write.writen.record__pushfn.perf_mmap__push
0.00 +4.7 4.68 ± 55% perf-profile.calltrace.cycles-pp.record__pushfn.perf_mmap__push.record__mmap_read_evlist.__cmd_record.cmd_record
0.00 +4.7 4.68 ± 55% perf-profile.calltrace.cycles-pp.write.writen.record__pushfn.perf_mmap__push.record__mmap_read_evlist
0.00 +4.7 4.68 ± 55% perf-profile.calltrace.cycles-pp.writen.record__pushfn.perf_mmap__push.record__mmap_read_evlist.__cmd_record
0.00 +4.9 4.90 ± 57% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +4.9 4.92 ± 26% perf-profile.calltrace.cycles-pp.sw_perf_event_destroy._free_event.perf_event_release_kernel.perf_release.__fput
0.00 +5.0 4.99 ±100% perf-profile.calltrace.cycles-pp.__intel_pmu_enable_all.perf_rotate_context.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt
0.00 +5.0 4.99 ±100% perf-profile.calltrace.cycles-pp.perf_rotate_context.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.00 +5.1 5.08 ±102% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.00 +5.1 5.14 ± 28% perf-profile.calltrace.cycles-pp.perf_mmap__push.record__mmap_read_evlist.__cmd_record.cmd_record.run_builtin
0.00 +5.1 5.14 ± 28% perf-profile.calltrace.cycles-pp.record__mmap_read_evlist.__cmd_record.cmd_record.run_builtin.handle_internal_command
0.00 +5.4 5.43 ± 25% perf-profile.calltrace.cycles-pp._free_event.perf_event_release_kernel.perf_release.__fput.task_work_run
0.00 +5.8 5.82 ± 94% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_do_entry
0.00 +5.8 5.82 ± 94% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt
0.00 +6.1 6.07 ± 90% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +6.6 6.62 ± 24% perf-profile.calltrace.cycles-pp.__cmd_record.cmd_record.run_builtin.handle_internal_command.main
0.00 +6.6 6.62 ± 24% perf-profile.calltrace.cycles-pp.cmd_record.run_builtin.handle_internal_command.main
0.00 +6.8 6.76 ± 18% perf-profile.calltrace.cycles-pp.exit_mmap.__mmput.exit_mm.do_exit.do_group_exit
0.00 +7.6 7.56 ± 76% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_do_entry.acpi_idle_enter
0.00 +8.0 8.03 ± 27% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +8.0 8.03 ± 27% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +8.0 8.05 ± 68% perf-profile.calltrace.cycles-pp.acpi_safe_halt.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter
0.00 +8.1 8.13 ± 28% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +8.1 8.13 ± 28% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
0.00 +8.1 8.13 ± 28% perf-profile.calltrace.cycles-pp.read
0.00 +9.1 9.05 ± 35% perf-profile.calltrace.cycles-pp.handle_internal_command.main
0.00 +9.1 9.05 ± 35% perf-profile.calltrace.cycles-pp.main
0.00 +9.1 9.05 ± 35% perf-profile.calltrace.cycles-pp.run_builtin.handle_internal_command.main
0.00 +9.3 9.26 ± 30% perf-profile.calltrace.cycles-pp.perf_event_release_kernel.perf_release.__fput.task_work_run.do_exit
0.00 +9.3 9.26 ± 30% perf-profile.calltrace.cycles-pp.perf_release.__fput.task_work_run.do_exit.do_group_exit
0.00 +10.1 10.14 ± 28% perf-profile.calltrace.cycles-pp.__fput.task_work_run.do_exit.do_group_exit.get_signal
0.00 +10.2 10.23 ± 27% perf-profile.calltrace.cycles-pp.task_work_run.do_exit.do_group_exit.get_signal.arch_do_signal_or_restart
0.00 +11.0 10.98 ± 55% perf-profile.calltrace.cycles-pp.asm_sysvec_reschedule_ipi.acpi_safe_halt.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state
0.00 +20.6 20.64 ± 30% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +20.6 20.64 ± 30% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
1.21 ± 3% +36.6 37.80 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
1.21 ± 3% +36.6 37.80 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.common_startup_64
1.22 ± 3% +36.8 38.00 ± 13% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.common_startup_64
1.22 ± 3% +36.9 38.10 ± 13% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.common_startup_64
1.22 ± 3% +36.9 38.10 ± 13% perf-profile.calltrace.cycles-pp.start_secondary.common_startup_64
1.21 ± 3% +37.2 38.43 ± 11% perf-profile.calltrace.cycles-pp.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
1.21 ± 3% +37.2 38.43 ± 11% perf-profile.calltrace.cycles-pp.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
1.21 ± 3% +37.3 38.54 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
1.22 ± 3% +37.6 38.84 ± 12% perf-profile.calltrace.cycles-pp.common_startup_64
2.19 ± 3% +53.9 56.10 ± 19% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state
95.60 -95.2 0.41 ±138% perf-profile.children.cycles-pp.__mmap
94.14 -93.7 0.49 ±130% perf-profile.children.cycles-pp.__mmap_new_vma
93.79 -93.5 0.31 ±134% perf-profile.children.cycles-pp.vma_link_file
93.40 -93.4 0.00 perf-profile.children.cycles-pp.rwsem_down_write_slowpath
93.33 -93.3 0.00 perf-profile.children.cycles-pp.rwsem_optimistic_spin
94.55 -93.1 1.42 ± 60% perf-profile.children.cycles-pp.ksys_mmap_pgoff
92.91 -92.9 0.00 perf-profile.children.cycles-pp.osq_lock
93.44 -92.7 0.75 ±109% perf-profile.children.cycles-pp.down_write
94.46 -92.6 1.84 ± 34% perf-profile.children.cycles-pp.vm_mmap_pgoff
94.45 -92.6 1.84 ± 34% perf-profile.children.cycles-pp.do_mmap
94.25 -92.6 1.66 ± 37% perf-profile.children.cycles-pp.__mmap_region
95.58 -44.8 50.78 ± 11% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
95.58 -44.8 50.78 ± 11% perf-profile.children.cycles-pp.do_syscall_64
0.00 +1.1 1.09 ± 33% perf-profile.children.cycles-pp.dup_mmap
0.00 +1.4 1.37 ± 49% perf-profile.children.cycles-pp.setlocale
0.00 +1.9 1.93 ± 26% perf-profile.children.cycles-pp.dup_mm
0.03 ± 70% +2.0 1.99 ± 36% perf-profile.children.cycles-pp.handle_softirqs
0.00 +2.0 1.99 ± 36% perf-profile.children.cycles-pp.__irq_exit_rcu
0.00 +2.0 2.02 ± 38% perf-profile.children.cycles-pp.folios_put_refs
0.00 +2.1 2.06 ± 52% perf-profile.children.cycles-pp._raw_spin_lock
0.00 +2.2 2.16 ± 44% perf-profile.children.cycles-pp.do_pte_missing
0.00 +2.2 2.21 ± 68% perf-profile.children.cycles-pp.link_path_walk
0.00 +2.2 2.23 ± 33% perf-profile.children.cycles-pp.copy_process
0.00 +2.3 2.30 ± 40% perf-profile.children.cycles-pp.__tlb_batch_free_encoded_pages
0.00 +2.3 2.30 ± 40% perf-profile.children.cycles-pp.free_pages_and_swap_cache
0.00 +2.3 2.34 ± 46% perf-profile.children.cycles-pp.walk_component
0.00 +2.4 2.37 ± 36% perf-profile.children.cycles-pp.zap_present_ptes
0.00 +2.5 2.48 ± 32% perf-profile.children.cycles-pp.get_cpu_sleep_time_us
0.00 +2.6 2.62 ± 35% perf-profile.children.cycles-pp.__do_sys_clone
0.00 +2.6 2.62 ± 35% perf-profile.children.cycles-pp.kernel_clone
0.00 +2.7 2.68 ± 35% perf-profile.children.cycles-pp.get_idle_time
0.00 +2.8 2.77 ± 33% perf-profile.children.cycles-pp.uptime_proc_show
0.00 +2.9 2.91 ± 32% perf-profile.children.cycles-pp._Fork
0.00 +3.1 3.10 ± 64% perf-profile.children.cycles-pp.proc_reg_read_iter
0.00 +3.2 3.24 ± 39% perf-profile.children.cycles-pp.bprm_execve
0.00 +3.2 3.24 ± 36% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.00 +3.2 3.24 ± 36% perf-profile.children.cycles-pp.x64_sys_call
0.00 +3.8 3.85 ± 39% perf-profile.children.cycles-pp.__x64_sys_execve
0.00 +3.8 3.85 ± 39% perf-profile.children.cycles-pp.do_execveat_common
0.00 +3.8 3.85 ± 39% perf-profile.children.cycles-pp.execve
0.00 +4.0 3.99 ± 38% perf-profile.children.cycles-pp.mutex_unlock
0.00 +4.2 4.19 ± 31% perf-profile.children.cycles-pp.zap_pte_range
0.00 +4.2 4.25 ± 65% perf-profile.children.cycles-pp.generic_perform_write
0.00 +4.3 4.29 ± 29% perf-profile.children.cycles-pp.unmap_page_range
0.00 +4.3 4.29 ± 29% perf-profile.children.cycles-pp.zap_pmd_range
0.00 +4.3 4.31 ± 51% perf-profile.children.cycles-pp.do_filp_open
0.00 +4.3 4.31 ± 51% perf-profile.children.cycles-pp.path_openat
0.19 ± 23% +4.4 4.60 ± 26% perf-profile.children.cycles-pp.__handle_mm_fault
0.00 +4.5 4.46 ± 59% perf-profile.children.cycles-pp.shmem_file_write_iter
0.00 +4.5 4.55 ± 24% perf-profile.children.cycles-pp.event_function_call
0.00 +4.5 4.55 ± 24% perf-profile.children.cycles-pp.smp_call_function_single
0.00 +4.6 4.58 ± 30% perf-profile.children.cycles-pp.unmap_vmas
0.51 ± 6% +4.6 5.14 ± 24% perf-profile.children.cycles-pp.handle_mm_fault
0.00 +4.7 4.68 ± 55% perf-profile.children.cycles-pp.record__pushfn
0.00 +4.7 4.68 ± 55% perf-profile.children.cycles-pp.writen
0.00 +4.8 4.80 ± 48% perf-profile.children.cycles-pp.do_sys_openat2
0.77 ± 3% +4.8 5.59 ± 21% perf-profile.children.cycles-pp.exc_page_fault
0.76 ± 3% +4.8 5.59 ± 21% perf-profile.children.cycles-pp.do_user_addr_fault
0.00 +4.9 4.90 ± 57% perf-profile.children.cycles-pp.ksys_write
0.00 +4.9 4.90 ± 57% perf-profile.children.cycles-pp.vfs_write
0.00 +4.9 4.90 ± 48% perf-profile.children.cycles-pp.__x64_sys_openat
0.00 +4.9 4.92 ± 26% perf-profile.children.cycles-pp.sw_perf_event_destroy
0.00 +5.0 4.99 ±100% perf-profile.children.cycles-pp.perf_rotate_context
0.00 +5.0 5.01 ± 54% perf-profile.children.cycles-pp.write
0.00 +5.1 5.09 ±102% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.00 +5.4 5.43 ± 25% perf-profile.children.cycles-pp._free_event
1.18 +5.6 6.78 ± 20% perf-profile.children.cycles-pp.asm_exc_page_fault
0.46 +5.6 6.07 ± 90% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.00 +5.7 5.75 ± 39% perf-profile.children.cycles-pp.perf_mmap__push
0.00 +5.7 5.75 ± 39% perf-profile.children.cycles-pp.record__mmap_read_evlist
0.53 +5.8 6.28 ± 89% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.53 +5.8 6.28 ± 89% perf-profile.children.cycles-pp.hrtimer_interrupt
0.00 +6.6 6.65 ± 77% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.00 +6.8 6.85 ± 20% perf-profile.children.cycles-pp.exit_mm
0.58 ± 2% +7.6 8.14 ± 75% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.00 +7.7 7.67 ± 23% perf-profile.children.cycles-pp.exit_mmap
0.00 +7.7 7.67 ± 30% perf-profile.children.cycles-pp.seq_read_iter
0.00 +7.7 7.72 ± 80% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
0.00 +7.8 7.75 ± 23% perf-profile.children.cycles-pp.__mmput
0.00 +8.0 8.03 ± 27% perf-profile.children.cycles-pp.ksys_read
0.00 +8.0 8.03 ± 27% perf-profile.children.cycles-pp.vfs_read
0.00 +8.1 8.13 ± 28% perf-profile.children.cycles-pp.read
0.02 ±141% +9.0 9.05 ± 35% perf-profile.children.cycles-pp.__cmd_record
0.02 ±141% +9.0 9.05 ± 35% perf-profile.children.cycles-pp.cmd_record
0.02 ±141% +9.0 9.05 ± 35% perf-profile.children.cycles-pp.handle_internal_command
0.02 ±141% +9.0 9.05 ± 35% perf-profile.children.cycles-pp.main
0.02 ±141% +9.0 9.05 ± 35% perf-profile.children.cycles-pp.run_builtin
0.00 +9.3 9.26 ± 30% perf-profile.children.cycles-pp.perf_event_release_kernel
0.00 +9.3 9.26 ± 30% perf-profile.children.cycles-pp.perf_release
1.02 ± 4% +9.3 10.33 ± 27% perf-profile.children.cycles-pp.task_work_run
0.00 +11.0 11.05 ± 28% perf-profile.children.cycles-pp.__fput
0.00 +15.8 15.85 ± 25% perf-profile.children.cycles-pp.arch_do_signal_or_restart
0.00 +15.8 15.85 ± 25% perf-profile.children.cycles-pp.get_signal
0.00 +19.1 19.09 ± 19% perf-profile.children.cycles-pp.do_exit
0.00 +19.1 19.09 ± 19% perf-profile.children.cycles-pp.do_group_exit
1.70 ± 2% +30.7 32.41 ± 21% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
1.22 ± 3% +36.9 38.10 ± 13% perf-profile.children.cycles-pp.start_secondary
1.21 ± 3% +37.2 38.43 ± 11% perf-profile.children.cycles-pp.acpi_idle_do_entry
1.21 ± 3% +37.2 38.43 ± 11% perf-profile.children.cycles-pp.acpi_idle_enter
1.21 ± 3% +37.2 38.43 ± 11% perf-profile.children.cycles-pp.acpi_safe_halt
1.22 ± 3% +37.3 38.54 ± 12% perf-profile.children.cycles-pp.cpuidle_idle_call
1.21 ± 3% +37.3 38.54 ± 12% perf-profile.children.cycles-pp.cpuidle_enter
1.21 ± 3% +37.3 38.54 ± 12% perf-profile.children.cycles-pp.cpuidle_enter_state
1.22 ± 3% +37.6 38.84 ± 12% perf-profile.children.cycles-pp.common_startup_64
1.22 ± 3% +37.6 38.84 ± 12% perf-profile.children.cycles-pp.cpu_startup_entry
1.22 ± 3% +37.6 38.84 ± 12% perf-profile.children.cycles-pp.do_idle
92.37 -92.4 0.00 perf-profile.self.cycles-pp.osq_lock
0.00 +2.1 2.06 ± 52% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +2.6 2.61 ± 36% perf-profile.self.cycles-pp.smp_call_function_single
0.00 +3.7 3.68 ± 37% perf-profile.self.cycles-pp.mutex_unlock
0.00 +6.6 6.65 ± 77% perf-profile.self.cycles-pp.__intel_pmu_enable_all
1.19 ± 3% +29.2 30.38 ± 15% perf-profile.self.cycles-pp.acpi_safe_halt
