On Thu, Jul 20, 2017 at 05:59:02PM +0300, Sagi Grimberg wrote: > Hey All, > > > REMINDER !!!!!!! > > I won't be able to attend, but I'd be very happy if a group > of smart people can discuss some random topics I had in mind: > > 1. Adding NUMA hints to *all* rdma resources. Right now > there is no way to ask the core to allocate a queue-pair on > a specific numa-node. Some drivers use the device home node > for memory allocation but application locality will usually be > much much better (sq/rq/cq spinlock cost *a lot* more if accessed > across numa nodes for example). > > Applications that don't care can simply skip the hint (or pass > NUMA_NO_NODE) and then drivers can do whatever they want. > > In just about any kernel driver, resource setup and IO path run on different > contexts, so numa locality is pretty much always wrong > (or right by accident). Its also true for a lot of user-space > applications. > > I also wander what is the least intrusive way to add it. > > > 2. SRQ sizing. No one knows what is a correct SRQ size. This leads > to magic random numbers for SRQ sizes (often copied) pretty much > anywhere used (usually crazy deep). Can the drivers hint applications > what is a "reasonable" size? Can it know? What if I want SRQ per core > or per numa-node? > > > 3. Adaptive interrupt moderation. In order to effectively use irq-poll > (or any other napi-like functionality), interrupt moderation is > important, but its pretty useless if its not adaptive (always too early or > too late). IMO It would be very useful to have, But before drivers > can start implementing it, we need a good core framework in place. > Does anyone think this is useful to have? > > > 4. I wander if there is any interest to have a striding receive queue > core API? It would be *very* useful to just about any existing kernel > ULP. Who wouldn't like to reduce the frequency of post receive > operations. Thanks Sagi, The topics look interesting, but it is unclear to me if ULP developers plan to attend LPC, due to SNIA timing. It is hard to discuss those hardcore topics without people who will actually use it. > > > Cheers, > Sagi.
Attachment:
signature.asc
Description: PGP signature
