Bart, > Submitting a copy operation as two bios or two requests means that > there is a risk that one of the two operations never reaches the block > driver at the bottom of the storage stack and hence that a deadlock > occurs. I prefer not to introduce any mechanisms that can cause a > deadlock. How do you copy a block range without offload? You perform a READ to read the data into memory. And once the READ completes, you do a WRITE of the data to the new location. Token-based copy offload works exactly the same way. You do a POPULATE TOKEN which is identical to a READ except you get a cookie instead of the actual data. And then once you have the cookie, you perform a WRITE USING TOKEN to perform the write operation. Semantically, it's exactly the same as a normal copy except for the lack of data movement. That's the whole point! Once I had support for token-based copy offload working, it became clear to me that this approach is much simpler than pointer matching, bio pairs, etc. The REQ_OP_COPY_IN operation and the REQ_OP_COPY_OUT operation are never in flight at the same time. There are no synchronization hassles, no lifetimes, no lookup tables in the sd driver, no nonsense. Semantically, it's a read followed by a write. For devices that implement single-command copy offload, the REQ_OP_COPY_IN operation only serves as a validation that no splitting took place. Once the bio reaches the ULD, the I/O is completed without ever sending a command to the device. blk-lib then issues a REQ_OP_COPY_OUT which gets turned into EXTENDED COPY or NVMe Copy and sent to the destination device. Aside from making things trivially simple, the COPY_IN/COPY_OUT semantic is a *requirement* for token-based offload devices. Why would we even consider having two incompatible sets of copy offload semantics coexist in the block layer? -- Martin K. Petersen Oracle Linux Engineering
