On 12/27/18 12:33 AM, James Smart wrote:
Once the IO buff allocations were made shared, there was a single
XRI buffer list shared by all hardware queues. A single list isn't
great for performance when shared across the per-cpu hardware queues.
Create a separate XRI IO buffer get/put list for each Hardware
Queue. As SGLs and associated IO buffers get allocated/posted to
the firmware; round robin their assignment across all available
hardware Queues so that there is an equitable assignment.
Modify SCSI and NVME IO submit code paths to use the Hardware Queue
logic for XRI allocation.
Add a debugfs interface to display hardware queue statistics
Added new empty_io_bufs counter to track if a cpu runs out of XRIs.
Replace common_ variables/names with io_ to make meanings clearer.
Have you looked at using embedded xri buffers?
Now that you have a common xri buffer structure it should be possible to
switch to embedded xri buffers, and rely on blk-mq sbitmap tag
allocation to manage the xri buffers.
Alternatively one could _identify_ the xri with the tag and have the xri
pool managed implicitly...
But either way would avoid this additional overhead, and we could use
the block-layer mechanisms directly.
Cheers,
Hannes
