Hello,
this is a preview series for the mcp251xfd which adds IRQ coalescing
support.
- What is IRQ coalescing and how does the mcp251xfd driver implement it?
The idea behind IRQ coalescing is to not serve every interrupt (CAN
frame RX'ed and CAN frame TX complete) as soon as possible, but to
delay it and handle several RX/TX complete frames at once. This
reduces the number of IRQs and SPI transfers.
With activated RX IRQ coalescing, the RX IRQ handler deactivated the
"RX FIFO not empty interrupt" and activated the "FIFO half full" (or
"FIFO full IRQ" - depending on configuration) instead.
To ensure that a single RX'ed CAN frame (which doesn't trigger the
FIFO half full IRQ) doesn't starve in the FIFO, a hrtimer is started
that activates the "FIFO not empty" IRQ after a configurable delay.
TX IRQ coalescing does the same thing, but for TX complete IRQs
- How to configure this?
Configuration is a bit tricky, it consists of several parameters,
which are all influencing each other and the number of buffers is
limited to power-of-two, to up to 32 per FIFO.
1) Configure the CAN mode (classical CAN-2.0 or CAN-FD) mode. Do not
bring the interface up.
2) Configure RX and TX FIFOs. In ethtool's speak this is called "ring"
configuration. The current ring configuration is shown with the
"-g" parameter:
| $ ethtool -g mcp251xfd1
|
| Ring parameters for mcp251xfd1:
| Pre-set maximums:
| RX: 96
| RX Mini: n/a
| RX Jumbo: n/a
| TX: 16
| Current hardware settings:
| RX: 80
| RX Mini: n/a
| RX Jumbo: n/a
| TX: 8
For TX, 1 FIFO is used with the default depth 8 (CAN-2.0 mode) and
4 (CAN-FD mode). In default configuration the driver uses the
remaining space for RX. In CAN-2.0 mode, this leads to 80 RX
buffers and 8 TX buffers. A more detailed overview is printed when
the interface is brought up:
| FIFO setup: TEF: 0x400: 8*12 bytes = 96 bytes
| FIFO setup: RX-0: FIFO 1/0x460: 32*20 bytes = 640 bytes
| FIFO setup: RX-1: FIFO 2/0x6e0: 32*20 bytes = 640 bytes
| FIFO setup: RX-2: FIFO 3/0x960: 16*20 bytes = 320 bytes
| FIFO setup: TX: FIFO 4/0xaa0: 8*16 bytes = 128 bytes
| FIFO setup: free: 224 bytes
Note:
- The number of RX buffers takes more priority than the number of
TX buffers.
- Ring configuration is reset by CAN mode configuration.
- Ring configuration resets coalescing configuration.
- Configuration is only possible if the interface is down.
Let's increase the number of RX buffers to the max of 96.
| $ ethtool -G mcp251xfd1 rx 96 tx 4
Check config with "-g":
| $ sudo ethtool -g mcp251xfd1
| Ring parameters for mcp251xfd1:
| Pre-set maximums:
| RX: 96
| RX Mini: n/a
| RX Jumbo: n/a
| TX: 16
| Current hardware settings:
| RX: 96
| RX Mini: n/a
| RX Jumbo: n/a
| TX: 4
The detailed output during ifup:
| FIFO setup: TEF: 0x400: 4*12 bytes = 48 bytes
| FIFO setup: RX-0: FIFO 1/0x430: 32*20 bytes = 640 bytes
| FIFO setup: RX-1: FIFO 2/0x6b0: 32*20 bytes = 640 bytes
| FIFO setup: RX-2: FIFO 3/0x930: 32*20 bytes = 640 bytes
| FIFO setup: TX: FIFO 4/0xbb0: 4*16 bytes = 64 bytes
| FIFO setup: free: 16 bytes
3) Configure the RX/TX IRQ coalescing.
The driver supports both RX and TX coalescing. The configuration is
done again with ethtool, the interface must be down for this.
There are 2 parameters to configure:
1) FIFO fill level that triggers IRQ
2) Delay after IRQ processing to enable FIFO not empty IRQ
In this example we configure RX coalescing for 32 buffers with a
delay of 10ms:
| $ ethtool -C mcp251xfd1 rx-usecs-irq 10000 rx-frames-irq 32
Check with "-c":
| $ ethtool -c mcp251xfd1
|
| Coalesce parameters for mcp251xfd1:
| Adaptive RX: n/a TX: n/a
| stats-block-usecs: n/a
| sample-interval: n/a
| pkt-rate-low: n/a
| pkt-rate-high: n/a
|
| rx-usecs: n/a
| rx-frames: n/a
| rx-usecs-irq: 10000
| rx-frames-irq: 32
|
| tx-usecs: n/a
| tx-frames: n/a
| tx-usecs-irq: 0
| tx-frames-irq: 1
|
| rx-usecs-low: n/a
| rx-frame-low: n/a
| tx-usecs-low: n/a
| tx-frame-low: n/a
|
| rx-usecs-high: n/a
| rx-frame-high: n/a
| tx-usecs-high: n/a
| tx-frame-high: n/a
The TX IRQ coalescing parameters we see in this example output are
"tx-usecs-irq=0" and "tx-frames-irq=1". This means no coalescing,
i.e. every TX complete event triggers an IRQ and is directly served
in the driver.
Note:
- Use "rx-usecs-irq=0" and "rx-frames-irq=1" to switch off RX
coalescing, accordingly for TX.
- Coalescing configuration is reset by ring configuration.
- Configuration is only possible if the interface is down.
Known issues:
- ifup with RX-coalescing configuration sometimes leads to RX stopping
after a few CAN frames on heavy loaded busses.
- Patches need proper description.
Happy testing. I'm especially interested in numbers regarding
reduction of system load and max CAN bus utilization. Further numbers
to look at is the number of IRQs from the mcp251xfd chip and number of
SPI host controller IRQs.
Note: small transfers on the rapi don't use IRQ mode per default, you
might see an increase of SPI IRQs if the driver reads a lot of CAN
frames from the FIFO in one transfer.
regards,
Marc
PS: The patches are available as a git branch on kernel.org:
https://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can-next.git/log/?h=mcp251xfd-coalesce
