Hey All,
REMINDER !!!!!!!
I won't be able to attend, but I'd be very happy if a group
of smart people can discuss some random topics I had in mind:
1. Adding NUMA hints to *all* rdma resources. Right now
there is no way to ask the core to allocate a queue-pair on
a specific numa-node. Some drivers use the device home node
for memory allocation but application locality will usually be
much much better (sq/rq/cq spinlock cost *a lot* more if accessed
across numa nodes for example).
Applications that don't care can simply skip the hint (or pass
NUMA_NO_NODE) and then drivers can do whatever they want.
In just about any kernel driver, resource setup and IO path run on
different contexts, so numa locality is pretty much always wrong
(or right by accident). Its also true for a lot of user-space
applications.
I also wander what is the least intrusive way to add it.
2. SRQ sizing. No one knows what is a correct SRQ size. This leads
to magic random numbers for SRQ sizes (often copied) pretty much
anywhere used (usually crazy deep). Can the drivers hint applications
what is a "reasonable" size? Can it know? What if I want SRQ per core
or per numa-node?
3. Adaptive interrupt moderation. In order to effectively use irq-poll
(or any other napi-like functionality), interrupt moderation is
important, but its pretty useless if its not adaptive (always too early
or too late). IMO It would be very useful to have, But before drivers
can start implementing it, we need a good core framework in place.
Does anyone think this is useful to have?
4. I wander if there is any interest to have a striding receive queue
core API? It would be *very* useful to just about any existing kernel
ULP. Who wouldn't like to reduce the frequency of post receive
operations.
Cheers,
Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
