Reply inline, with a good bit of snipping done (posting via gmane, so quote/content ratio is an issue). Andy Grover wrote: > +These backstores cover the most common use cases, but not all. One new > +use case that other non-kernel target solutions, such as tgt, are able > +to support is using Gluster's GLFS or Ceph's RBD as a backstore. The > +target then serves as a translator, allowing initiators to store data > +in these non-traditional networked storage systems, while still only > +using standard protocols themselves. Another use case is in supporting various image formats, like (say) qcow2, and then handing those off to vhost_scsi. > +Benefits: > + > +In addition to allowing relatively easy support for RBD and GLFS, TCMU > +will also allow easier development of new backstores. TCMU combines > +with the LIO loopback fabric to become something similar to FUSE > +(Filesystem in Userspace), but at the SCSI layer instead of the > +filesystem layer. A SUSE, if you will. As long as people don't start calling it L[UNs in ]USER[space] :P Between that and ABUSE (A Block device in USErspace), this domain has some real naming potential... > +Device Discovery: > + > +Other devices may be using UIO besides TCMU. Unrelated user processes > +may also be handling different sets of TCMU devices. TCMU userspace > +processes must find their devices by scanning sysfs > +class/uio/uio*/name. For TCMU devices, these names will be of the > +format: > + > +tcm-user/<subtype>/<path> > + > +where "tcm-user" is common for all TCMU-backed UIO devices. <subtype> > +will be a userspace-process-unique string to identify the TCMU device > +as expecting to be backed by a certain handler, and <path> will be an > +additional handler-specific string for the user process to configure > +the device, if needed. Neither <subtype> or <path> can contain ':', > +due to LIO limitations. It might be good to change this somewhat; in the vast majority of cases it'd be saner for userspace programs to figure this information out via udev etc. rather than parsing sysfs themselves. This information is still worth documenting, but saying things like "must find their devices by scanning sysfs" is likely to lead to users of this interface making suboptimal choices. > +Device Events: > + > +If a new device is added or removed, user processes will recieve a HUP > +signal, and should re-scan sysfs. File descriptors for devices no > +longer in sysfs should be closed, and new devices should be opened and > +handled. Is there a cleaner way to do this? In particular, re-scanning sysfs may cause race conditions (device removed, one of the same name re-added but a different UIO device node; probably more to be found). Perhaps recommend netlink uevents, so that remove+add is noticeable? Also, is the SIGHUP itself the best option? Could we simply require the user process to listen for add/remove uevents to get such change notifications, and thus enforce good behavior? > +Writing a user backstore handler: > + > +Variable emulation with pass_level: > + > +TCMU supports a "pass_level" option with valid values of 1, 2, or > +3. This controls how many different SCSI commands are passed up, > +versus being emulated by LIO. The purpose of this is to give the user > +handler author a choice of how much of the full SCSI command set they > +care to support. > + > +At level 1, only READ and WRITE commands will be seen. At level 2, > +additional commands defined in the SBC SCSI specification such as > +WRITE SAME, SYNCRONIZE CACHE, and UNMAP will be passed up. Finally, at > +level 3, almost all commands defined in the SPC SCSI specification > +will also be passed up for processing by the user handler. One use case I'm actually interested in is having userspace provide something other than just SPC - for instance, tgt can provide a virtual tape library or an OSD, and CDemu can provide emulated optical discs from various image formats. Currently, CDemu uses its own out-of-tree driver called VHBA (Virtual Host Bus Adapter) to do pretty much exactly what TCMU+Loopback would accomplish... and in the process misses out on all of the other fabrics, unless you're willing to _re-import_ those devices using PSCSI, which has its own quirks. Perhaps there could be a level 0 (or 4, or whatever) which means "explicitly enabled list of commands" - maybe as a bitmap that could be passed to the kernel somehow? Hopefully, that could also avoid some of the quirks of PSCSI regarding ALUA and such - if it's not implemented, leave the relevant bits at zero, and LIO handles it. This does look really nice, thanks for writing it! -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
