Hello!
Recently, we tried to migrate from the kernel 4.19.125 to 5.4.17.
During stress tests we observe that on the same hardware (we are using
virtual machines in our on-prem cloud on VMWare) and using the same
XDP application, on the 4.19.125 softirq CPU utilization is 23% vs.
the 35% on 5.4.17. Please, note, we are using the FIPS on 5.4.17
kernel.
At the first glance, using linux perf I see some difference, but can't
understand them:
5.4.17 perf top -a
11.90% [kernel] [k] lookup_nulls_elem_raw
8.79% [kernel] [k] __do_softirq
4.97% [kernel] [k]
_raw_spin_unlock_irqrestore
3.62% [kernel] [k] memcpy_orig
3.39% bpf_prog_2e039a5b937c8960 [k] bpf_prog_2e039a5b937c8960
2.66% [kernel] [k] native_write_msr
2.34% [kernel] [k] vmxnet3_rq_rx_complete
1.87% [kernel] [k] arch_local_irq_restore
1.71% [kernel] [k] __htab_map_lookup_elem
1.33% [kernel] [k] page_frag_free
1.31% [kernel] [k] kmem_cache_free
1.28% [kernel] [k] vmxnet3_poll_rx_only
1.13% [kernel] [k] finish_task_switch
4.19.125 perf top -a
12.90% [kernel] [k] lookup_nulls_elem_raw
9.11% [kernel] [k] __softirqentry_text_start
4.36% [kernel] [k]
_raw_spin_unlock_irqrestore
3.66% [kernel] [k] memcpy_orig
3.03% [kernel] [k] native_write_msr
2.69% [kernel] [k] vmxnet3_rq_rx_complete
2.34% [kernel] [k] _lock+0x800000005cc2
1.91% [kernel] [k] arch_local_irq_restore
1.53% [kernel] [k] __htab_map_lookup_elem
1.36% [kernel] [k] __slab_free
1.31% [kernel] [k] vmxnet3_poll_rx_only
1.24% [kernel] [k] page_frag_free
1.14% [kernel] [k] finish_task_switch
Could you help me please to identify softirq CPU utilization increase?
--
Alexander Petrovsky
|