While browsing this months archive (prior to posting my question
on SCSI device renumbering), I noticed a question regarding QLogics
HBAs and MPIO configuration.
My experience has been with QLA2200 adapters from IBM (IBM FAStT HBA). The java tool they ship to aid in configuring these adapters is known as FAStT MSJ (Managment Suite Java). One of my biggest gripes in this project is the dearth of good documentation. (I am really beginning to think SAN vendors are being obtuse just so they can sell consulting/implementation services.)
However, I did find a bit of a gem which may give us some clues on how to manually configure the qla2x00 drivers. There is an Appendix in an IBM RedBoot entitled "Linux with xSeries and FAStT: Essentials" SG24-7026-00. Appendix B, Figure 5-23. Also, adding to the knowledge base are a few notes from the IBM FAStT HBA Linux device driver (v6.06.60) README file. All these tidbits pertain to options passed to the qla2x00 drivers.
Here are the ones I've discovered:
ConfigRequired -- unclear, all stuff I've seen seem to pass
"ConfigRequired=1" to the driver.ql2xmaxqdepth -- max. queue depth to report for target devices
ql2xretrycount -- Max. number of mid-layer retries allowed for
a command. The default value for failover
mode is 30, for non-failover mode is 20.
(When using IBM FAStT Storage Servers this
must be set to 60.)"failover mode" ==> MPIO support is enabled in the driver.
ql2xsuspendcount -- Number of 6-sec. suspend iterations to perform
while a target returns "NOT READY" status. The
default is 10 (which is equal to 60s.). (This
value must be set to 40 for IBM FAStT Storage
Servers.) displayConfig -- If set to "1", then the loaded configuration
will be displayed. (Look in syslog, e.g.
/var/log/messages.) max_srbs -- Max. number of simultaneous commands allowed
for an HBA. The default is set to 4096 in
the driver source code. ql2xopts -- When doing MPIO stuff, this will contain an
obnoxiously long string with info about port
and node WWN's, preferred paths, etc. See
below for more on this.A simple set of configuration options that I've currently placed in my /etc/modules.conf is:
options qla2200 displayConfig=1 ql2xretrycount=60 ql2xsuspendcount=40
A more complex example might be (from the afore-mentioned RedBook).
options qla2200 displayConfig=1 ConfigRequired=1
qla2xopts=scsi-qla0-adapter-port=210000e08b059fa1\;scsi-qla0-
tgt-0-di-0-node=200200a0b80cbf7e\;scsi-qla0-tgt-0-di-0-port=2
00200a0b80cbf7f\;scsi-qla0-tgt-0-di-0-pid=0000ef\;scsi-qla0-t
gt-0-di-0-preferred=fffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffff5\;scsi-qla0-tgt-0-di-0-control=00\;scs
i-qla1-adapter-port=210000e08b057ea2\;scsi-qla1-tgt-0-di-1-no
de=200200a0b80cbf7e\;scsi-qla1-tgt-0-di-1-port=200300a0b80cbf
7f\;scsi-qla1-tgt-0-di-1-pid=0000e4\;scsi-qla1-tgt-0-di-1-pre
ferred=000000000000000000000000000000000000000000000000000000
000000000a\;scsi-qla1-tgt-0-di-1-control=80\;
TTBOMK: * this must all be on one line * there are no whitespace characters after the "=" sign in qla2xopts=... * all semi-colons are escaped with a backslash * those long strings of hex digits are exactly 64 hex digits, and describe a 256-bit mask * all of the above is for a single "disk" with two available paths (more evidence for this assumption in Fig 5-37 in the Redbook)
Now, trying to infer what it all means. I would really appreciate any feedback that you all might have.
1) There are two HBA adapters: "qla0" and "qla1".
2) The adapter port ID's can be overridden
HBA #1: scsi-qla0-adapter-port = 210000e08b059fa1 HBA #2: scsi-qla1-adapter-port = 210000e08b057ea2
From the device driver readme file (para-phrased):
| scsi-qla<#>-adapter-port=<adapter_port_name_value> | | -- specifies the FC port name to be used for the adapter. | Where <adapter_port_name_value> is the FC port name value in | hex. The default value is port name found in the adapter's | NVRAM.
3) Each SCSI ID can be associated with a specific WWN. Hence,
if we assume HBA #1 is the 2nd SCSI controller (scsi1) in
the server, and HBA #2 is the 3rd (scsi2), then I think the
following phrases are interpreted as:
scsi-qla0-tgt-0-di-0-node=200200a0b80cbf7e
scsi-qla1-tgt-0-di-1-node=200200a0b80cbf7e
The two paths scsi(1,0,0,0) and scsi(2,0,0,1) are both associated with the same SAN "disk", identified by WWN 200200a0b80cbf7e.
From the dev. driver README (para-phrased):
| scsi-qla<Q>-tgt-<N>-di-<I>-node=<device_FC_name> | | -- associates specified FC device with SCSI target ID "<N>" and | device id "<I>" on qla2x00 instance number "<Q>"
(Note - I'm assuming that "I" is a SCSI LUN here.)
4) ... and the SCSI ID can be associated with a port (i.e. path) for getting to the specific device (node) specfied in the last paragraph.
scsi-qla0-tgt-0-di-0-port=200200a0b80cbf7f
scsi-qla1-tgt-0-di-1-port=200300a0b80cbf7f
From the dev. driver README (para-phrased):
| scsi-qla<Q>-tgt-<N>-di-<I>-port=<port_FC_name> | | -- associates specified FC port with SCSI target ID "<N>" and | device id "<I>" on qla2x00 instance number "<Q>"
5) ... any guesses?
scsi-qla0-tgt-0-di-0-pid=0000ef scsi-qla1-tgt-0-di-1-pid=0000e4
6) This looks like where the preferred path is specified:
scsi-qla0-tgt-0-di-0-preferred=ffff....fff5
scsi-qla1-tgt-0-di-1-preferred=0000....000aI'm not sure how to interpret the bits. I can that the numbers are complementary. I haven't found any more clues though (I guess the next level would be the driver source).
7) ... any guesses?
scsi-qla0-tgt-0-di-0-control=00
scsi-qla1-tgt-0-di-1-control=80
8) And, finally, one last documented parameter from the driver source README file:
| scsi-qla<Q>-tgt-<N>-di-<I>-diabled=<256_bit_mask> | | -- associates specified mask with SCSI target ID "<N>" and | device id "<I>" on qla2x00 instance number "<Q>". | | The 256-bit mask must be exactly 64 hex digits. | | Using the mask [MSB] 00000000....0000000f [LSB] will cause | the first four LUNs [3, 2, 1, and 0] of a given target to be | disabled on that target/path. |
I have found no examples which used the last one.
Anyway, that's a digest of things I've dug up this last week. If anyone else
can help shed light on this, I'd surely appreciate it.
- Don
_______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
