SelvaKumar, > Add new BLKCOPY ioctl that offloads copying of multiple sources > to a destination to the device. Your patches are limited in scope to what is currently possible with NVMe. I.e. multiple source ranges to a single destination within the same device. That's fine, I think the garbage collection use case is valid and worth pursuing. I just wanted to go over what the pain points were for the various attempts in SCSI over the years. The main headache was due the stacking situation with DM and MD. Restricting offload to raw SCSI disks would have been simple but not really a good fit for most real world developments that often use DM or MD to provision the storage. Things are simple for DM/MD with reads and writes because you have one bio as parent that may get split into many clones that complete individually prior to the parent being marked as completed. In the copy offload scenario things quickly become complex once both source and destination ranges have to be split into multiple commands for potentially multiple devices. And these clones then need to be correctly paired at the bottom of the stack. There's also no guarantee that a 1MB source range maps to a single 1MB destination range. So you could end up with an M:N relationship to resolve. After a few failed attempts we focused on single source range/single destination range. Just to simplify the slicing and dicing. That worked reasonably well. However, then came along the token-based commands in SCSI and those threw a wrench in the gears. Now the block layer plumbing had to support two completely different semantic approaches. Inspired by a combination of Mikulas' efforts with pointer matching and the token-based approach in SCSI I switched the block layer implementation from a single operation (REQ_COPY) to something similar to the SCSI token approach with a REQ_COPY_IN and a REQ_COPY_OUT. The premise being that you would send a command to the source device and "get" the data. In the EXTENDED COPY scenario, the data wasn't really anything but a confirmation from the SCSI disk driver that the I/O had reached the bottom of the stack without being split by DM/MD. And once completion of the REQ_COPY_IN reached blk-lib, a REQ_COPY_OUT would be issued and, if that arrived unchanged in the disk driver, get turned into an EXTENDED COPY sent to the destination. In the token-based scenario the same thing happened except POPULATE TOKEN was sent all the way out to the device to receive a cookie representing the source block ranges. Upon completion, that cookie was used by blk-lib to issue a REQ_COPY_OUT command which was then sent to the destination device. Again only if the REQ_COPY_OUT I/O hadn't been split traversing the stack. The idea was to subsequently leverage the separation of REQ_COPY_IN and REQ_COPY_OUT to permit a DM/MD iterative approach to both stages of the operation. That seemed to me like the only reasonable way to approach the M:N splitting problem (if at all)... -- Martin K. Petersen Oracle Linux Engineering
