On 2016-08-11 19:15:40 [+0300], Grygorii Strashko wrote: > Mark CPSW Rx/Tx IRQs as IRQF_NO_THREAD and avoid double scheduling on -RT > where this IRQs are forced threaded: > rx-irq > |- schedule threaded rx-irq handler > ... > |- threaded rx-irq handler -> cpsw_rx_interrupt() > |- napi_schedule() > |- __raise_softirq_irqoff() > |- wakeup_proper_softirq() > ... > napi This should not be the default path. The default should be napi running in the context of the threaded rx-irq handler once the handler is done. The wakeup_proper_softirq() part is only done if napi thinks that the callback functions runs for too long. So in *that* case you continue NAPI in the softirq-thread which runs at SCHED_OTHER. > after: > rx-irq > |- cpsw_rx_interrupt() > |- napi_schedule() > |- irq_exit() > |- invoke_softirq() > |- wakeup_softirqd() > ... > napi Since you schedule the softirq from an IRQ-off region / without a process context you force the softirq to run in the thread at SCHED_OTHER priority. > And, as result, get benefits from the following improvements (tested > on am57xx-evm): > > 1) "[ 78.348599] NOHZ: local_softirq_pending 80" message will not be > seen any more. Now these warnings can be seen once iperf is started. > # iperf -c $IPERFHOST -w 128K -d -t 60 Do you also see "sched: RT throttling activated"? Because I don't see otherwise why this should pop up. > 2) latency reduction when cyclictest is run in parallel with network load > where net_perf.sh is: > iperf -c $IPERFHOST -w 8K -d -t 60 > iperf -c $IPERFHOST -w 16K -d -t 60 > iperf -c $IPERFHOST -w 32K -d -t 60 > iperf -c $IPERFHOST -w 64K -d -t 60 > iperf -c $IPERFHOST -w 128K -d -t 60 > > before: > T: 0 ( 1326) P:98 I:1000 C: 240000 Min: 8 Act: 13 Avg: 18 Max: 70 > T: 1 ( 1327) P:98 I:1500 C: 159981 Min: 9 Act: 15 Avg: 16 Max: 43 > after: > T: 0 ( 1331) P:98 I:1000 C: 240000 Min: 8 Act: 15 Avg: 14 Max: 51 > T: 1 ( 1332) P:98 I:1500 C: 159953 Min: 8 Act: 16 Avg: 15 Max: 33 -d 0 to have I: set to the same value. What does -i 250 say? And without network load we are where we were at "after" values? What happens if s/__raise_softirq_irqoff_ksoft/__raise_softirq_irqoff/ in net/core/dev.c and chrt the priority of you network interrupt handlers to SCHED_OTHER priority? > 3) network performance increase > > win, K Mbits/s > before after % > 8K 354 350.3 0.0 > 16K 412 551 33.7 > 32K 423 659.5 55.9 > 64K 436 728.3 67.0 > 128K 537 845 57.4 How close are the after numbers to !RT? Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
