Move the sas readme to the Documents dir Signed-off-by: Alexis Bruemmer <alexisb@xxxxxxxxxx> Signed-off-by: Mike Anderson <andmike@xxxxxxxxxx> --- drivers/scsi/sas/README | 484 --------------------------------------------- Documentation/scsi/sas.txt | 3 2 files changed, 3 insertions(+), 484 deletions(-) Index: aic94xx-sas-2.6-patched/Documentation/scsi/sas.txt =================================================================== --- /dev/null +++ aic94xx-sas-2.6-patched/Documentation/scsi/sas.txt @@ -0,0 +1,3 @@ +SAS Layer +--------- +Coming Soon Index: aic94xx-sas-2.6-patched/drivers/scsi/sas/README =================================================================== --- aic94xx-sas-2.6-patched.orig/drivers/scsi/sas/README +++ /dev/null @@ -1,484 +0,0 @@ -SAS Layer ---------- - -The SAS Layer is a management infrastructure which manages -SAS LLDDs. It sits between SCSI Core and SAS LLDDs. The -layout is as follows: while SCSI Core is concerned with -SAM/SPC issues, and a SAS LLDD+sequencer is concerned with -phy/OOB/link management, the SAS layer is concerned with: - - * SAS Phy/Port/HA event management (LLDD generates, - SAS Layer processes), - * SAS Port management (creation/destruction), - * SAS Domain discovery and revalidation, - * SAS Domain device management, - * SCSI Host registration/unregistration, - * Device registration with SCSI Core (SAS) or libata - (SATA), and - * Expander management and exporting expander control - to user space. - -A SAS LLDD is a PCI device driver. It is concerned with -phy/OOB management, and vendor specific tasks and generates -events to the SAS layer. - -The SAS Layer does most SAS tasks as outlined in the SAS 1.1 -spec. - -The sas_ha_struct describes the SAS LLDD to the SAS layer. -Most of it is used by the SAS Layer but a few fields need to -be initialized by the LLDDs. - -After initializing your hardware, from the probe() function -you call sas_register_ha(). It will register your LLDD with -the SCSI subsystem, creating a SCSI host and it will -register your SAS driver with the sysfs SAS tree it creates. -It will then return. Then you enable your phys to actually -start OOB (at which point your driver will start calling the -notify_* event callbacks). - -Structure descriptions: - -struct sas_phy -------------------- -Normally this is statically embedded to your driver's -phy structure: - struct my_phy { - blah; - struct sas_phy sas_phy; - bleh; - }; -And then all the phys are an array of my_phy in your HA -struct (shown below). - -Then as you go along and initialize your phys you also -initialize the sas_phy struct, along with your own -phy structure. - -In general, the phys are managed by the LLDD and the ports -are managed by the SAS layer. So the phys are initialized -and updated by the LLDD and the ports are initialized and -updated by the SAS layer. - -There is a scheme where the LLDD can RW certain fields, -and the SAS layer can only read such ones, and vice versa. -The idea is to avoid unnecessary locking. - -enabled -- must be set (0/1) -id -- must be set [0,MAX_PHYS) -class, proto, type, role, oob_mode, linkrate -- must be set -oob_mode -- you set this when OOB has finished and then notify -the SAS Layer. - -sas_addr -- this normally points to an array holding the sas -address of the phy, possibly somewhere in your my_phy -struct. - -attached_sas_addr -- set this when you (LLDD) receive an -IDENTIFY frame or a FIS frame, _before_ notifying the SAS -layer. The idea is that sometimes the LLDD may want to fake -or provide a different SAS address on that phy/port and this -allows it to do this. At best you should copy the sas -address from the IDENTIFY frame or maybe generate a SAS -address for SATA directly attached devices. The Discover -process may later change this. - -frame_rcvd -- this is where you copy the IDENTIFY/FIS frame -when you get it; you lock, copy, set frame_rcvd_size and -unlock the lock, and then call the event. It is a pointer -since there's no way to know your hw frame size _exactly_, -so you define the actual array in your phy struct and let -this pointer point to it. You copy the frame from your -DMAable memory to that area holding the lock. - -sas_prim -- this is where primitives go when they're -received. See sas.h. Grab the lock, set the primitive, -release the lock, notify. - -port -- this points to the sas_port if the phy belongs -to a port -- the LLDD only reads this. It points to the -sas_port this phy is part of. Set by the SAS Layer. - -ha -- may be set; the SAS layer sets it anyway. - -lldd_phy -- you should set this to point to your phy so you -can find your way around faster when the SAS layer calls one -of your callbacks and passes you a phy. If the sas_phy is -embedded you can also use container_of -- whatever you -prefer. - - -struct sas_port -------------------- -The LLDD doesn't set any fields of this struct -- it only -reads them. They should be self explanatory. - -phy_mask is 32 bit, this should be enough for now, as I -haven't heard of a HA having more than 8 phys. - -lldd_port -- I haven't found use for that -- maybe other -LLDD who wish to have internal port representation can make -use of this. - - -struct sas_ha_struct -------------------- -It normally is statically declared in your own LLDD -structure describing your adapter: -struct my_sas_ha { - blah; - struct sas_ha_struct sas_ha; - struct my_phy phys[MAX_PHYS]; - struct sas_port sas_ports[MAX_PHYS]; /* (1) */ - bleh; -}; - -(1) If your LLDD doesn't have its own port representation. - -What needs to be initialized (sample function given below). - -pcidev -sas_addr -- since the SAS layer doesn't want to mess with - memory allocation, etc, this points to statically - allocated array somewhere (say in your host adapter - structure) and holds the SAS address of the host - adapter as given by you or the manufacturer, etc. -sas_port -sas_phy -- an array of pointers to structures. (see - note above on sas_addr). - These must be set. See more notes below. -num_phys -- the number of phys present in the sas_phy array, - and the number of ports present in the sas_port - array. There can be a maximum num_phys ports (one per - port) so we drop the num_ports, and only use - num_phys. - -The event interface: - - /* LLDD calls these to notify the class of an event. */ - void (*notify_ha_event)(struct sas_ha_struct *, enum ha_event); - void (*notify_port_event)(struct sas_phy *, enum port_event); - void (*notify_phy_event)(struct sas_phy *, enum phy_event); - -When sas_register_ha() returns, those are set and can be -called by the LLDD to notify the SAS layer of such events -the SAS layer. - -The port notification: - - /* The class calls these to notify the LLDD of an event. */ - void (*lldd_port_formed)(struct sas_phy *); - void (*lldd_port_deformed)(struct sas_phy *); - -If the LLDD wants notification when a port has been formed -or deformed it sets those to a function satisfying the type. - -A SAS LLDD should also implement at least one of the Task -Management Functions (TMFs) described in SAM: - - /* Task Management Functions. Must be called from process context. */ - int (*lldd_abort_task)(struct sas_task *); - int (*lldd_abort_task_set)(struct domain_device *, u8 *lun); - int (*lldd_clear_aca)(struct domain_device *, u8 *lun); - int (*lldd_clear_task_set)(struct domain_device *, u8 *lun); - int (*lldd_I_T_nexus_reset)(struct domain_device *); - int (*lldd_lu_reset)(struct domain_device *, u8 *lun); - int (*lldd_query_task)(struct sas_task *); - -For more information please read SAM from T10.org. - -Port and Adapter management: - - /* Port and Adapter management */ - int (*lldd_clear_nexus_port)(struct sas_port *); - int (*lldd_clear_nexus_ha)(struct sas_ha_struct *); - -A SAS LLDD should implement at least one of those. - -Phy management: - - /* Phy management */ - int (*lldd_control_phy)(struct sas_phy *, enum phy_func); - -lldd_ha -- set this to point to your HA struct. You can also -use container_of if you embedded it as shown above. - -A sample initialization and registration function -can look like this (called last thing from probe()) -*but* before you enable the phys to do OOB: - -static int register_sas_ha(struct my_sas_ha *my_ha) -{ - int i; - static struct sas_phy *sas_phys[MAX_PHYS]; - static struct sas_port *sas_ports[MAX_PHYS]; - - my_ha->sas_ha.sas_addr = &my_ha->sas_addr[0]; - - for (i = 0; i < MAX_PHYS; i++) { - sas_phys[i] = &my_ha->phys[i].sas_phy; - sas_ports[i] = &my_ha->sas_ports[i]; - } - - my_ha->sas_ha.sas_phy = sas_phys; - my_ha->sas_ha.sas_port = sas_ports; - my_ha->sas_ha.num_phys = MAX_PHYS; - - my_ha->sas_ha.lldd_port_formed = my_port_formed; - - my_ha->sas_ha.lldd_dev_found = my_dev_found; - my_ha->sas_ha.lldd_dev_gone = my_dev_gone; - - my_ha->sas_ha.lldd_max_execute_num = lldd_max_execute_num; (1) - - my_ha->sas_ha.lldd_queue_size = ha_can_queue; - my_ha->sas_ha.lldd_execute_task = my_execute_task; - - my_ha->sas_ha.lldd_abort_task = my_abort_task; - my_ha->sas_ha.lldd_abort_task_set = my_abort_task_set; - my_ha->sas_ha.lldd_clear_aca = my_clear_aca; - my_ha->sas_ha.lldd_clear_task_set = my_clear_task_set; - my_ha->sas_ha.lldd_I_T_nexus_reset= NULL; (2) - my_ha->sas_ha.lldd_lu_reset = my_lu_reset; - my_ha->sas_ha.lldd_query_task = my_query_task; - - my_ha->sas_ha.lldd_clear_nexus_port = my_clear_nexus_port; - my_ha->sas_ha.lldd_clear_nexus_ha = my_clear_nexus_ha; - - my_ha->sas_ha.lldd_control_phy = my_control_phy; - - return sas_register_ha(&my_ha->sas_ha); -} - -(1) This is normally a LLDD parameter, something of the -lines of a task collector. What it tells the SAS Layer is -whether the SAS layer should run in Direct Mode (default: -value 0 or 1) or Task Collector Mode (value greater than 1). - -In Direct Mode, the SAS Layer calls Execute Task as soon as -it has a command to send to the SDS, _and_ this is a single -command, i.e. not linked. - -Some hardware (e.g. aic94xx) has the capability to DMA more -than one task at a time (interrupt) from host memory. Task -Collector Mode is an optional feature for HAs which support -this in their hardware. (Again, it is completely optional -even if your hardware supports it.) - -In Task Collector Mode, the SAS Layer would do _natural_ -coalescing of tasks and at the appropriate moment it would -call your driver to DMA more than one task in a single HA -interrupt. DMBS may want to use this by insmod/modprobe -setting the lldd_max_execute_num to something greater than -1. - -(2) SAS 1.1 does not define I_T Nexus Reset TMF. - -Events ------- - -Events are _the only way_ a SAS LLDD notifies the SAS layer -of anything. There is no other method or way a LLDD to tell -the SAS layer of anything happening internally or in the SAS -domain. - -Phy events: - PHYE_LOSS_OF_SIGNAL, (C) - PHYE_OOB_DONE, - PHYE_OOB_ERROR, (C) - PHYE_SPINUP_HOLD. - -Port events, passed on a _phy_: - PORTE_BYTES_DMAED, (M) - PORTE_BROADCAST_RCVD, (E) - PORTE_LINK_RESET_ERR, (C) - PORTE_TIMER_EVENT, (C) - PORTE_HARD_RESET. - -Host Adapter event: - HAE_RESET - -A SAS LLDD should be able to generate - - at least one event from group C (choice), - - events marked M (mandatory) are mandatory (only one), - - events marked E (expander) if it wants the SAS layer - to handle domain revalidation (only one such). - - Unmarked events are optional. - -Meaning: - -HAE_RESET -- when your HA got internal error and was reset. - -PORTE_BYTES_DMAED -- on receiving an IDENTIFY/FIS frame -PORTE_BROADCAST_RCVD -- on receiving a primitive -PORTE_LINK_RESET_ERR -- timer expired, loss of signal, loss -of DWS, etc. (*) -PORTE_TIMER_EVENT -- DWS reset timeout timer expired (*) -PORTE_HARD_RESET -- Hard Reset primitive received. - -PHYE_LOSS_OF_SIGNAL -- the device is gone (*) -PHYE_OOB_DONE -- OOB went fine and oob_mode is valid -PHYE_OOB_ERROR -- Error while doing OOB, the device probably -got disconnected. (*) -PHYE_SPINUP_HOLD -- SATA is present, COMWAKE not sent. - -(*) should set/clear the appropriate fields in the phy, - or alternatively call the inlined sas_phy_disconnected() - which is just a helper, from their tasklet. - -The Execute Command SCSI RPC: - - int (*lldd_execute_task)(struct sas_task *, int num, - unsigned long gfp_flags); - -Used to queue a task to the SAS LLDD. @task is the tasks to -be executed. @num should be the number of tasks being -queued at this function call (they are linked listed via -task::list), @gfp_mask should be the gfp_mask defining the -context of the caller. - -This function should implement the Execute Command SCSI RPC, -or if you're sending a SCSI Task as linked commands, you -should also use this function. - -That is, when lldd_execute_task() is called, the command(s) -go out on the transport *immediately*. There is *no* -queuing of any sort and at any level in a SAS LLDD. - -The use of task::list is two-fold, one for linked commands, -the other discussed below. - -It is possible to queue up more than one task at a time, by -initializing the list element of struct sas_task, and -passing the number of tasks enlisted in this manner in num. - -Returns: -SAS_QUEUE_FULL, -ENOMEM, nothing was queued; - 0, the task(s) were queued. - -If you want to pass num > 1, then either -A) you're the only caller of this function and keep track - of what you've queued to the LLDD, or -B) you know what you're doing and have a strategy of - retrying. - -As opposed to queuing one task at a time (function call), -batch queuing of tasks, by having num > 1, greatly -simplifies LLDD code, sequencer code, and _hardware design_, -and has some performance advantages in certain situations -(DBMS). - -The LLDD advertises if it can take more than one command at -a time at lldd_execute_task(), by setting the -lldd_max_execute_num parameter (controlled by "collector" -module parameter in aic94xx SAS LLDD). - -You should leave this to the default 1, unless you know what -you're doing. - -This is a function of the LLDD, to which the SAS layer can -cater to. - -int lldd_queue_size - The host adapter's queue size. This is the maximum -number of commands the lldd can have pending to domain -devices on behalf of all upper layers submitting through -lldd_execute_task(). - -You really want to set this to something (much) larger than -1. - -This _really_ has absolutely nothing to do with queuing. -There is no queuing in SAS LLDDs. - -struct sas_task { - dev -- the device this task is destined to - list -- must be initialized (INIT_LIST_HEAD) - task_proto -- _one_ of enum sas_proto - scatter -- pointer to scatter gather list array - num_scatter -- number of elements in scatter - total_xfer_len -- total number of bytes expected to be transfered - data_dir -- PCI_DMA_... - task_done -- callback when the task has finished execution -}; - -When an external entity, entity other than the LLDD or the -SAS Layer, wants to work with a struct domain_device, it -_must_ call kobject_get() when getting a handle on the -device and kobject_put() when it is done with the device. - -This does two things: - A) implements proper kfree() for the device; - B) increments/decrements the kref for all players: - domain_device - all domain_device's ... (if past an expander) - port - host adapter - pci device - and up the ladder, etc. - -DISCOVERY ---------- - -The sysfs tree has the following purposes: - a) It shows you the physical layout of the SAS domain at - the current time, i.e. how the domain looks in the - physical world right now. - b) Shows some device parameters _at_discovery_time_. - -This is a link to the tree(1) program, very useful in -viewing the SAS domain: -ftp://mama.indstate.edu/linux/tree/ -I expect user space applications to actually create a -graphical interface of this. - -That is, the sysfs domain tree doesn't show or keep state if -you e.g., change the meaning of the READY LED MEANING -setting, but it does show you the current connection status -of the domain device. - -Keeping internal device state changes is responsibility of -upper layers (Command set drivers) and user space. - -When a device or devices are unplugged from the domain, this -is reflected in the sysfs tree immediately, and the device(s) -removed from the system. - -The structure domain_device describes any device in the SAS -domain. It is completely managed by the SAS layer. A task -points to a domain device, this is how the SAS LLDD knows -where to send the task(s) to. A SAS LLDD only reads the -contents of the domain_device structure, but it never creates -or destroys one. - -Expander management from User Space ------------------------------------ - -In each expander directory in sysfs, there is a file called -"smp_portal". It is a binary sysfs attribute file, which -implements an SMP portal (Note: this is *NOT* an SMP port), -to which user space applications can send SMP requests and -receive SMP responses. - -Functionality is deceptively simple: - -1. Build the SMP frame you want to send. The format and layout - is described in the SAS spec. Leave the CRC field equal 0. -open(2) -2. Open the expander's SMP portal sysfs file in RW mode. -write(2) -3. Write the frame you built in 1. -read(2) -4. Read the amount of data you expect to receive for the frame you built. - If you receive different amount of data you expected to receive, - then there was some kind of error. -close(2) -All this process is shown in detail in the function do_smp_func() -and its callers, in the file "expander_conf.c". - -The kernel functionality is implemented in the file -"sas_expander.c". - -The program "expander_conf.c" implements this. It takes one -argument, the sysfs file name of the SMP portal to the -expander, and gives expander information, including routing -tables. - -The SMP portal gives you complete control of the expander, -so please be careful. - : send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html