On 10/31/2024 7:48 PM, Joe Damato wrote:
Describe irq suspension, the epoll ioctls, and the tradeoffs of using
different gro_flush_timeout values.
Signed-off-by: Joe Damato <jdamato@xxxxxxxxxx>
Co-developed-by: Martin Karsten <mkarsten@xxxxxxxxxxxx>
Signed-off-by: Martin Karsten <mkarsten@xxxxxxxxxxxx>
Acked-by: Stanislav Fomichev <sdf@xxxxxxxxxxx>
Reviewed-by: Bagas Sanjaya <bagasdotme@xxxxxxxxx>
---
<snip>
+
+IRQ suspension
+--------------
+
+IRQ suspension is a mechanism wherein device IRQs are masked while epoll
+triggers NAPI packet processing.
+
+While application calls to epoll_wait successfully retrieve events, the kernel will
+defer the IRQ suspension timer. If the kernel does not retrieve any events
+while busy polling (for example, because network traffic levels subsided), IRQ
+suspension is disabled and the IRQ mitigation strategies described above are
+engaged.
+
+This allows users to balance CPU consumption with network processing
+efficiency.
+
+To use this mechanism:
+
+ 1. The per-NAPI config parameter ``irq_suspend_timeout`` should be set to the
+ maximum time (in nanoseconds) the application can have its IRQs
+ suspended. This is done using netlink, as described above. This timeout
+ serves as a safety mechanism to restart IRQ driver interrupt processing if
+ the application has stalled. This value should be chosen so that it covers
+ the amount of time the user application needs to process data from its
+ call to epoll_wait, noting that applications can control how much data
+ they retrieve by setting ``max_events`` when calling epoll_wait.
+
+ 2. The sysfs parameter or per-NAPI config parameters ``gro_flush_timeout``
+ and ``napi_defer_hard_irqs`` can be set to low values. They will be used
+ to defer IRQs after busy poll has found no data.
Is it required to set gro_flush_timeout and napi_defer_hard_irqs when
irq_suspend_timeout is set? Doesn't it override any smaller
gro_flush_timeout value?
+
+ 3. The ``prefer_busy_poll`` flag must be set to true. This can be done using
+ the ``EPIOCSPARAMS`` ioctl as described above.
+
+ 4. The application uses epoll as described above to trigger NAPI packet
+ processing.
+
+As mentioned above, as long as subsequent calls to epoll_wait return events to
+userland, the ``irq_suspend_timeout`` is deferred and IRQs are disabled. This
+allows the application to process data without interference.
+
+Once a call to epoll_wait results in no events being found, IRQ suspension is
+automatically disabled and the ``gro_flush_timeout`` and
+``napi_defer_hard_irqs`` mitigation mechanisms take over.
+
+It is expected that ``irq_suspend_timeout`` will be set to a value much larger
+than ``gro_flush_timeout`` as ``irq_suspend_timeout`` should suspend IRQs for
+the duration of one userland processing cycle.
.. _threaded:
