On Mon, 2008-02-04 at 13:24 -0800, Linus Torvalds wrote: > > On Mon, 4 Feb 2008, J. Bruce Fields wrote: > > > > I'd assumed the move was primarily because of the difficulty of getting > > correct semantics on a shared filesystem > > .. not even shared. It was hard to get correct semantics full stop. > > Which is a traditional problem. The thing is, the kernel always has some > internal state, and it's hard to expose all the semantics that the kernel > knows about to user space. > > So no, performance is not the only reason to move to kernel space. It can > easily be things like needing direct access to internal data queues (for a > iSCSI target, this could be things like barriers or just tagged commands - > yes, you can probably emulate things like that without access to the > actual IO queues, but are you sure the semantics will be entirely right? > > The kernel/userland boundary is not just a performance boundary, it's an > abstraction boundary too, and these kinds of protocols tend to break > abstractions. NFS broke it by having "file handles" (which is not > something that really exists in user space, and is almost impossible to > emulate correctly), and I bet the same thing happens when emulating a SCSI > target in user space. > > Maybe not. I _rally_ haven't looked into iSCSI, I'm just guessing there > would be things like ordering issues. > <nod>. The iSCSI CDBs and write immediate, unsoliciated, or soliciated data payloads may be received out of order across communication paths (which may be going over different subnets) within the nexus, but the execution of the CDB to SCSI Target Port must be in the same order as it came down from the SCSI subsystem on the initiator port. In iSCSI and iSER terms, this is called Command Sequence Number (CmdSN) ordering, and is enforced within each nexus. The initiator node will be assigning the CmdSNs as the CDBs come down, and when communication paths fail, unacknowledged CmdSNs will be retried on a different communication path when using iSCSI/iSER connection recovery. Already acknowledged CmdSNs will be explictly retried using a iSCSI specific task management function called TASK_REASSIGN. This along with CSM-I and CSM-E statemachines are collectly known as ErrorRecoveryLevel=2 in iSCSI. Anyways, here is a great visual of a modern iSCSI Target processor and SCSI Target Engine. The CmdSN ordering is representd by the oval across across iSCSI connections going to various network portals groups on the left side of the diagram. Thanks Eddy Q! http://www.haifa.il.ibm.com/satran/ips/EddyQuicksall-iSCSI-in-diagrams/portal_groups.pdf --nab - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
