On 1/28/19 5:40 PM, Christoph Hellwig wrote:
On Mon, Jan 28, 2019 at 03:33:46PM +0100, Hannes Reinecke wrote:
Well ... not always.
Some drivers (eg aacraid or hpsa) use internal commands to query hardware,
handle events and the like.
These commands use the same infrastructure than normal SCSI commands,
and hence need to use the same tag pool. But they are most definitely _not_
SCSI commands, and won't be needing any of those allocations.
They aren't scsi commands, but they need a very similar infrastructure,
and to facilitate code reuse we absolutely have to use the same data
structures, everything else is madness.
Okay. Which seems to point into the direction of implementing something
like blk_mq_get_reserved_rq().
We actually have a few uses like that in existing old SCSI drivers,
where we create a fake struct scsi_device to send command to the host,
which doesn't sound all that bad except for the fact that we need an
escape for the lun value to avoid getting in the way.
In general I'm not sure this is the most common use case - I'd expect
the most common use to be proper implementing TMFs..
Command abort and device reset being the most common, indeed, and can be
handled by creating an additional 'admin' queue.
Abort and device reset go to the logical unit, so there is no need
for any new case. And please avoid the name admin queue, it has a very
specific meaning in NVMe that doesn't translate easily to SCSI.
Hmm. True.
Not sure if it's worth separating the TMF from the configuration issue,
but might be worthwhile looking at.
The NVMe admin queue has an entirely separate tag pool and hardware
queue structure. Any sort of per-host queue in SCSI HBAs would
still share the tag pool and hardware queue infrastructure with the
I/O queues.
Out of necessity, yes.
And, in fact, having a shared tag pool is the sole reason why we even
have to discuss things :-)
The more interesting cases will be where internal commands are used to
retrieve configuration information. If we were to go with the admin queue
approach we'll need to reconfigure the tagset after issuing those commands.
Possible, but not entirely trivial.
Do we have an example for that?
aacraid.
It's using a per-hba array for internal commands (->fibs), which are
used for every command being send to the hardware.
There are two routines (aac_fib_alloc_tag and aac_fib_alloc) which
allocates command structures from the same pool. There are 8 fibs set
aside for management purposes, and 19 invocations of aac_fib_alloc() for
sending management commands. Only 3 of those invocations are for TMF (in
drivers/scsi/aacraid/linit.c); the remaining are for configuration and
event handling.
And it requires internal commands to figure out the HBA configuration
before the SCSI host is initiatlized and the tagset is created.
(cf aac_get_adapter_info(), aac_get_config_status(), and
aac_get_containers(); all are called before scsi_add_host())
Cheers,
Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
hare@xxxxxxx +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
