On Mon, 1 Jul 2019 15:44:29 +0100, Laatz, Kevin wrote: > On 28/06/2019 21:29, Jonathan Lemon wrote: > > On 28 Jun 2019, at 9:19, Laatz, Kevin wrote: > >> On 27/06/2019 22:25, Jakub Kicinski wrote: > >>> I think that's very limiting. What is the challenge in providing > >>> aligned addresses, exactly? > >> The challenges are two-fold: > >> 1) it prevents using arbitrary buffer sizes, which will be an issue > >> supporting e.g. jumbo frames in future. > >> 2) higher level user-space frameworks which may want to use AF_XDP, > >> such as DPDK, do not currently support having buffers with 'fixed' > >> alignment. > >> The reason that DPDK uses arbitrary placement is that: > >> - it would stop things working on certain NICs which need the > >> actual writable space specified in units of 1k - therefore we need 2k > >> + metadata space. > >> - we place padding between buffers to avoid constantly > >> hitting the same memory channels when accessing memory. > >> - it allows the application to choose the actual buffer size > >> it wants to use. > >> We make use of the above to allow us to speed up processing > >> significantly and also reduce the packet buffer memory size. > >> > >> Not having arbitrary buffer alignment also means an AF_XDP driver > >> for DPDK cannot be a drop-in replacement for existing drivers in > >> those frameworks. Even with a new capability to allow an arbitrary > >> buffer alignment, existing apps will need to be modified to use that > >> new capability. > > > > Since all buffers in the umem are the same chunk size, the original > > buffer > > address can be recalculated with some multiply/shift math. However, > > this is > > more expensive than just a mask operation. > > Yes, we can do this. That'd be best, can DPDK reasonably guarantee the slicing is uniform? E.g. it's not desperate buffer pools with different bases? > Another option we have is to add a socket option for querying the > metadata length from the driver (assuming it doesn't vary per packet). > We can use that information to get back to the original address using > subtraction. Unfortunately the metadata depends on the packet and how much info the device was able to extract. So it's variable length. > Alternatively, we can change the Rx descriptor format to include the > metadata length. We could do this in a couple of ways, for example, > rather than returning the address as the start of the packet, instead > return the buffer address that was passed in, and adding another 16-bit > field to specify the start of packet offset with that buffer. If using > another 16-bits of the descriptor space is not desirable, an alternative > could be to limit umem sizes to e.g. 2^48 bits (256 terabytes should be > enough, right :-) ) and use the remaining 16 bits of the address as a > packet offset. Other variations on these approach are obviously possible > too. Seems reasonable to me..
