Kernel check for pending softirqs periodically, they are performed in a few points of kernel code, such as irq_exit() and __local_bh_enable_ip(), softirqs that have been activated by a given CPU must be executed on the same CPU, this characteristic of softirq is always a potentially "dangerous" operation, because one CPU might be end up very busy while the other are most idle. Above concern is proven in a networking user case: recenlty, we engineer find out the time used for connection re-establishment on kernel v5.10 is 300 times larger than v4.19, meanwhile, softirq monopolize almost 99% of CPU. This problem stem from that the connection between Sender and Receiver node get lost, the NIC driver on Sender node will keep raising NET_TX softirq before connection recovery. The system log show that most of softirq is performed from __local_bh_enable_ip(), since __local_bh_enable_ip is used widley in kernel code, it is very easy to run out most of CPU, and the user-mode application can't obtain enough CPU cycles to establish connection as soon as possible. Although kernel limit the running time of __do_softirq(), it does not control the running time of entire softirqs on given CPU, so this patchset introduce a safeguard mechanism that allows the system administrator to allocate bandwidth for used by softirqs, this safeguard mechanism is known as Sofitrq Throttling and is controlled by two parameters in the /proc file system: /proc/sys/kernel/sofitrq_period_ms Defines the period in ms(millisecond) to be considered as 100% of CPU bandwidth, the default value is 1,000 ms(1second). Changes to the value of the period must be very well thought out, as too long or too short are beyond one's expectation. /proc/sys/kernel/softirq_runtime_ms Define the bandwidth available to softirqs on each CPU, the default values is 950 ms(0.95 second) or, in other words, 95% of the CPU bandwidth. Setting negative integer to this value means that softirqs my use up to 100% CPU times. The default values for softirq throttling mechanism define that 95% of the CPU time can be used by softirqs. The remaing 5% will be devoted to other kinds of tasks, such as syscall, interrupt, exception, real-time processes and normal processes when the softirqs workload in system are very heavy. System administrator can tune above two parameters to satifies the need of system performance and stability. Liao Chang (3): softirq: Add two parameters to control CPU bandwidth for use by softirq softirq: Do throttling when softirqs use up its bandwidth softirq: Introduce statistics about softirq throttling fs/proc/softirqs.c | 18 +++++ include/linux/interrupt.h | 7 ++ include/linux/kernel_stat.h | 27 +++++++ init/Kconfig | 10 +++ kernel/softirq.c | 155 ++++++++++++++++++++++++++++++++++++ kernel/sysctl.c | 16 ++++ 6 files changed, 233 insertions(+) -- 2.17.1