Nicholas A. Bellinger wrote:
On Wed, 2008-08-27 at 21:56 +0400, Vladislav Bolkhovitin wrote:
Our userspace tools will also be able to support both kernel space and
userspace targets with little trouble so distros that have stgt will not
notice any differences.
So this is one of the items that I mentioned to Vlad as being a
potential stumbling block.. I stated that I have no problem dumping the
LIO IOCTL in favor of something better, and having to break backwards
compat with LIO userspace tools (well, until I have to emulate the
backwards compat wrt target-ctl ;-)) is something I am ready to deal
with.
So I definately see your point here, having the exist CLI logic in place
into distros is something that we are going to have to deal with (even
if the core behind them is completely different).
I should also mention that virtually all of the configuration logic done
from the LIO-Core and LIO-Target IOCTLs today is intended (minus a bug
or two :-) to be done without having to shutdown/stop anything other
than the referenced object for the target-ctl operation. Obviously, I
really want to keep the same functionality with the CLI interface the
LIO pieces are ported to.
I will be sure to take a look at the STGT control interface and see how
I can fit the current functionality in. I believe SCST uses an IOCTL
today as well, which would put them in a similar situation.
I, personally, don't like an interface, like tgtadm, which tries to do
all non-trivial configuration work from a single command line tool,
because of the following 3 reasons:
1. It's a lot of effort to write and maintain such a tool, because it
needs to be extensible to work with new modules and modes. For instance,
iptables tool is 86K lines long. The whole netfilter code for all
network protocols in the kernel is less that 50K lines long. Do we want
such a code bloat (170% of code to configure) and dedicated team of
maintainers to solve a fundamentally simple configuration task?
Hrrmmm, yes I could see the same type of problem with the amount of
modularity requires for having plugins registering their own sets of
parameters for their control path code..
I should add in this context that in the UNIX world the choice between a
huge application, capable of doing all possible kinds of work, and a set
of small utilities, doing exactly one simple thing each, was done
decades ago in favor of the set of small utilities: find, grep, sort,
etc. And I believe now nobody would claim that it was a wrong choice.
2. It assumes the stateless type of configuration, when each call
configures exactly one thing without any side effects on already
configured or future entries.
This approach is good for cases like
iptables, but for SCSI targets it's possible that several configuration
steps require to be done in an atomic manner, like adding an iSCSI
target and configuring its parameters.
Well, the ability for an admin to force an LIO-Core related action, say
removing an HBA and all associated storage object with lots of exported
LUNs and running I/O, or an LIO-Target related action, say removing an
entire iSCSI Target Node with targetname=) at any time.. This obviously
require precision interaction between Target Fabric I/O Paths <-> Target
Core and Target Core <-> Control Interface to Admin.
That control interface needs to be protected in object contexts. In
LIO-Core this is on a per HBA (be it physical or virtual) context. With
LIO-Target this is on a iSCSI Target Node by targetname -> Target Portal
Group Tag context. Obviously doing this from an IOCTL is the only real
choice I had when this code started in 2001, but I wonder how configfs
would work for something like this.
My favorite configuration interface would have 2 levels.
1. The lowest level would be /proc, /sys, etc. based and allow to
configure exactly one parameter or set of related parameters with
corresponding subparameters necessary to provide required atomicity, if
needed. For instance, this is how a new virtual read-only SBC device
with 4K block size is added in SCST vdisk handler:
# echo "open disk_4K /disk_4K 4096 READ_ONLY" >/proc/scst/vdisk/vdisk
2. The higher level interface(s) would allow people to not bother with
low level commands, but use regular text config file(s). See scstadmin
utility for example. It allows to do all necessary configuration of SCST
core from /etc/scst.conf file. Such interface must have an important
property: it must be able to detect changes in the config file and apply
them to the running system. That property would allow to have system
configuration always persistent: if one needs to change something, he
would edit the config file and rerun the corresponding utility
(scstadmin in this example; it really can do that, though with some
limitations). Although this interface level would completely belong to
user space, we in kernel need to provide for it a convenient interface.
Target drivers and backstorage device handlers, who need advanced
configuration, would have own low and high level interfaces, as needed.
For instance, an iSCSI target must not start serving clients until all
its targets fully configured. Otherwise, initiators can get rejected for
not yet configured target and erroneously consider it dead. In
iSCSI-SCST the user space part of the target doesn't start accepting
connection until it finishes reading /etc/iscsi-scst.conf file.
3. It's hard to read 5+ parameters in one command line, so it's a lot
easier to make a mistake there.
No, I completely agree. But I honestly think the actual target CLI
interface and parameters to admin need to do alot of pre-execution
script logic in userspace to reference different interested objects,
without the admin have to provide all of stuff. I do this today to
determine the major/minor for lvm_uuid= (from lvs -v), md_uuid= (from
mdadm -D) and udev_path= (from /dev/disk)..
Same goes for real SCSI devices that we are exporting directly from
drivers/scsi. We want to use EVPD Unit Serial or Device Identificaton
where ever able to reference said storage object.
Yes, this is why we need the high level interface. Otherwise for complex
targets the configuration task quickly grows up to a nightmare.
So, I believe, a configuration interface should be rather /proc or /sys
interface based. I don't think we should care much about backward
compatibility with tgtadm, because the existing interface doesn't reach
the state of being widely used.
I would definately vote against proc here for the fancy stuff I
mentioned above. I have experience enabled core-iscsi to use sysfs for
RO data, but nothing along the lines of what would be required for a
generic target mode RW control path. Does anyone with sysfs experience
have any comments on thing..?
Sysfs as well as configfs have one big disadvantage. They limit each
file to only 4KB. This would force us for to create a subdirectory for
each device and for each connected initiator. I don't like seeing
thousands subdirectories. Additionally, such layout is a lot less
convenient for parsing for the high level configuration tool, which
needs to find out the difference between the current configuration and
content of the corresponding config file.
Currently, with procfs SCST can list in /proc/scst/sessions virtually
unlimited amount of connected initiators in a simple for parsing manner.
It was done using seq interface well and simply. Neither sysfs, nor
configfs support seq interface. This would introduce significant effort
in both kernel and user spaces.
Debugfs supports seq interface, but, because of the name, I doubt we can
use it ;)
Thus, looks like we'd better stay with /proc. After all, networking and
VM widely use /proc for internal configuration. Why SCSI target is worse?
As I already mentioned, only ibmvscsi at
the moment uses it, hardware for which is pretty rare.
I am sure we could get some time this hardware if we wanted.. :-)
Thus, I would suggest that before making the further move we should also
consider configuration interfaces of SCST and LIO and choose the best
things from all 3.
So, Ming mentioned configfs for this... /me reads about configfs
--nab
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
