On 8/20/21 3:39 AM, Kanchan Joshi wrote:
Bart, Mikulas
On Tue, Aug 17, 2021 at 10:44 PM Bart Van Assche <bvanassche@xxxxxxx> wrote:
On 8/17/21 3:14 AM, SelvaKumar S wrote:
Introduce REQ_OP_COPY, a no-merge copy offload operation. Create
bio with control information as payload and submit to the device.
Larger copy operation may be divided if necessary by looking at device
limits. REQ_OP_COPY(19) is a write op and takes zone_write_lock when
submitted to zoned device.
Native copy offload is not supported for stacked devices.
Using a single operation for copy-offloading instead of separate
operations for reading and writing is fundamentally incompatible with
the device mapper. I think we need a copy-offloading implementation that
is compatible with the device mapper.
While each read/write command is for a single contiguous range of
device, with simple-copy we get to operate on multiple discontiguous
ranges, with a single command.
That seemed like a good opportunity to reduce control-plane traffic
(compared to read/write operations) as well.
With a separate read-and-write bio approach, each source-range will
spawn at least one read, one write and eventually one SCC command. And
it only gets worse as there could be many such discontiguous ranges (for
GC use-case at least) coming from user-space in a single payload.
Overall sequence will be
- Receive a payload from user-space
- Disassemble into many read-write pair bios at block-layer
- Assemble those (somehow) in NVMe to reduce simple-copy commands
- Send commands to device
We thought payload could be a good way to reduce the
disassembly/assembly work and traffic between block-layer to nvme.
How do you see this tradeoff? What seems necessary for device-mapper
usecase, appears to be a cost when device-mapper isn't used.
Especially for SCC (since copy is within single ns), device-mappers
may not be too compelling anyway.
Must device-mapper support be a requirement for the initial support atop SCC?
Or do you think it will still be a progress if we finalize the
user-space interface to cover all that is foreseeable.And for
device-mapper compatible transport between block-layer and NVMe - we
do it in the later stage when NVMe too comes up with better copy
capabilities?
Hi Kanchan,
These days there might be more systems that run the device mapper on top
of the NVMe driver or a SCSI driver than systems that do use the device
mapper. It is common practice these days to use dm-crypt on personal
workstations and laptops. LVM (dm-linear) is popular because it is more
flexible than a traditional partition table. Android phones use
dm-verity on top of hardware encryption. In other words, not supporting
the device mapper means that a very large number of use cases is
excluded. So I think supporting the device mapper from the start is
important, even if that means combining individual bios at the bottom of
the storage stack into simple copy commands.
Thanks,
Bart.
