Hi,
Right now there is no generic interface that can be used by the
in-kernel components such as file-system or userspace application
(except passthru commands or some combination of write/read/compare) to
issue verify command with the central block layer API for device LBA
verification. This can lead to each userspace applications having
protocol specific IOCTL and that defeats the purpose of having the OS
provide to H/W abstraction.
I've worked on a patch series to introduce a new block layer payloadless
request operation REQ_OP_VERIFY that allows in-kernel components &
userspace applications to verify the range of the LBAs by offloading
checksum scrubbing/verification to the controller that is directly
attached to the host. For direct attached devices this leads to decrease
in the host DMA traffic and CPU usage and for the fabrics attached
devices over the network that leads to a decrease in the network traffic
and CPU usage for both host & target.
The main advantage is to free up the CPU and reduce the host link
traffic since for some devices, their internal bandwidth is higher than
the host link and offloading this operation can improve the performance
of the proactive error detection applications such as file system level
scrubbing.
The prominent example of this is xfs_scrub that issues SCSI Verify
command at the time of scrubbing that guarantees the LBA verification
is been done on the device level at the time of file system scrubbing.
Here are the results :-
When verifying 500GB of data on NVMeOF with nvme-loop and null_blk
as a target back-end block device results show almost a 80%
performance increase :-
With Verify resulting in REQ_OP_VERIFY to null_blk :-
real 2m3.773s
user 0m0.000s
sys 0m59.553s
With Emulation resulting in REQ_OP_READ null_blk :-
real 12m18.964s
user 0m0.002s
sys 1m15.666s
There is an objection from on this work to be merged :-
1. Current Storage protocols may lie.
2. We cannot trust any SSD vendors that they will implement Verify
command authentically.
3. Hence we should never rely on this support and kernel should
not have this functionality.
4. Instead a new command HASH is proposed.
I'd like to discuss the this topic in great detail and understand
what API we can create or protocol changes needed so we can upstream
this support.
-ck
