On 12/28/18 8:55 PM, James Smart wrote:
On 12/28/2018 1:10 AM, Hannes Reinecke wrote:
Have you looked at using embedded xri buffers?
Now that you have a common xri buffer structure it should be possible
to switch to embedded xri buffers, and rely on blk-mq sbitmap tag
allocation to manage the xri buffers.
Alternatively one could _identify_ the xri with the tag and have the
xri pool managed implicitly...
But either way would avoid this additional overhead, and we could use
the block-layer mechanisms directly.
Well, the XRI shouldn't map to a blk layer element as it will move on an
as-needed basis to ios for scsi (any of the sdevs) and for nvme (any of
the controllers and queues/elements on a queue). And currently, there's
a dma-mapped SGL that goes with the XRI that takes a mailbox command to
set up. There are some other "buffer" areas that could map, so it would
be useful. But for the time being, we stabilized on keeping this group
together.
Which was basically my point.
As the earlier patches moved things to have use a shared xri structure
for both NVMe and SCSI it should be possible to identify the blk-mq
'tag' with an XRI, and rely on the block layer to manage free tags.
But, given that we did see the buffers that want to be split out, we
have started a task to split them, use the io stack elements if
possible, and figure out a way to do SGL lists that is a little more
accommodating. The XRIs unfortunately will still need to exist in some
manner like they do after these patch sets as there's only so many to go
around.
Precisely. But the block-layer should now be able to handle shared tags
properly.
In summary - I agree with you, but it will be a later patch that gets us
closer to this and the xri mgmt routines put in by the patch set will be
needed regardless.
Yeah; for the above idea to work out we would need to have a working
'private' tag handling, which ATM is lacking for the SCSI part.
And we'd need to check if the block layer really is capable of having
shared tags between subsystems.
So it rather sounds like a topic for LSF (if and when that will happen).
Hence:
Reviewed-by: Hannes Reinecke <hare@xxxxxxxx>
Cheers,
Hannes
